redirecting

 redirecting...


cover

cover | next page >

Cover

title | : | Coding Theory and Cryptography : The Essentials Monographs and Textbooks in Pure and Applied Mathematics ; 234
author | : | Hankerson, Darrel R.
publisher | : | CRC Press
isbn10 | asin | : | 0824704657
print isbn13 | : | 9780824704650
ebook isbn13 | : | 9780585421414
language | : | English
subject |  | Coding theory, Cryptography.
publication date | : | 2000
lcc | : | QA268.C69 2000eb
ddc | : | 003/.54
subject | : | Coding theory, Cryptography.
cover | next page >


page_1

< previous page | page_1 | next page >

Page 1

Chapter 1
Introduction to Coding Theory

1.1 Introduction

Coding theory is the study of methods for efficient and accurate transfer of information from one place to another. The theory has been developed for such diverse applications as the minimization of noise from compact disc recordings, the transmission of financial information across telephone lines, data transfer from one computer to another or from memory to the central processor, and information transmission from a distant source such as a weather or communications satellite or the Voyager spacecraft which sent pictures of Jupiter and Saturn to Earth.

The physical medium through which the information is transmitted is called a channel. Telephone lines and the atmosphere are examples of channels. Undesirable disturbances, called noise, may cause the information received to differ from what was transmitted. Noise may be caused by sunspots, lightning, folds in a magnetic tape, meteor showers, competing telephone messages, random radio disturbance, poor typing, poor hearing, poor speech, or many other things.

Coding theory deals with the problem of detecting and correcting transmission errors caused by noise on the channel. The following diagram provides a rough idea of a general information transmission system.

The most important part of the diagram, as far as we are concerned, is the noise, for without it there would be no need for the theory.

In practice, the control we have over this noise is the choice of a good channel to use for transmission and the use of various noise filters to combat certain types of interference which may be encountered. These are engineering problems. Once we have settled on the best mechanical system for solving these problems, we can

< previous page | page_1 | next page >


page_10

< previous page | page_10 | next page >

Page 10

1.6.9 Which of the codewords 110110, 110101, 000111, 100111, 101000 is most likely to have been sent if w = 011001 is received.

1.6.10 In Theorem 1.6.3 we assume that 1/2 < p < 1. What would change in the statement of Theorem 1.6.3 if we replace this assumption with

(a) 0 < p < 1/2,

(b) p = 1/2?

1.7 Some basic algebra

A problem that we shall need to address is that of finding an efficient way of finding the closest codeword to any received word. If the code has many codewords then it is impractical to compare each received word w to each codeword in turn to find which codeword disagrees with w in as few positions as possible. For example, if the code contains 212 codewords (as was used for the Voyager mission) then such a decoding procedure could never hope to keep up with the incoming transmission. To overcome this problem, we need to introduce some structure into our codes.

Let K = {0,1} and let Kn be the set of all binary words of length n. Define addition and multiplication of the elements of K as follows:

0+0 = 0,0+1 = 1,1+0 = 1,1+1 = 0

0·0 = 0,1·0 = 0,0·1 = 0,1·1 = 1.

Define addition for the elements of Kn componentwise, using the addition defined on K to add each component. For example, let

v = 01101 and w = 11001 then v+w = 10100.

Clearly the addition of two binary words of length n results in a binary word of length n, so Kn is closed under addition.

Using linear algebra terminology, we refer to an element of K as scalar. Then scalar multiplication of Kn is defined componentwise. Since the only scalars are 0 and 1, the only scalar multiples of a word w are 0·w, which is the element of Kn with 0 in every component, and 1. w, which is w. We refer to the element of Kn with 0 in all components as the zero word. Clearly Kn is closed under scalar multiplication.

With these definitions of addition and scalar multiplication, it can be shown that Kn is a vector space. That is, for any words of length n, u, v and w and for any scalars a and b:

1. v + w ∈ Kn

2. (u + v) + w = u + (v + w)

3. v+0 = 0+v = v, where 0 is the zero word
< previous page | page_10 | next page >


page_100

< previous page | page_100 | next page >

Page 100

word | polynomial f(x) | factorization h(x)g(x) of f(x)
000000 | 0 | 0(1 + x 3)
100100 | 1 + x 3 | 1(1 + x 3)
010010 | x + x 4 | x(1 + x 3)
001001 | x 2 + x 5 | x 2(1 + x 3)
110110 | 1 + x + x 3 + x 4 | (1 + x)(1 + x 3)
101101 | 1 + x 2 + x 3 + x 5 | (1 + x 2)(1 + x 3)
011011 | x + x 2 + x 4 + x 5 | (x + x 2)(1 + x 3)
111111 | 1 + x + x 2 + x 3 + x 4 + x 5 | (1 + x + x 2)(1 + x 3)
Table 4.2: Codewords are multiples of the generator polynomial.

Theorem 4.2.17 g(x) is the generator polynomial for a linear cyclic code of length n if and only if g(x) divides 1 + xn (so 1 + xn = h(x)g(x)).

Proof: By Lemma 4.2.12, c(x) = h(x)g(x) (mod 1 + xn) = h(x)g(x) + q(x)(1 + xn), is a codeword for any h(x). By the Division Algorithm, g(x) will divide every codeword c(x) if and only if it divides 1 + xn. Hence by Theorem 4.2.13 g(x) is the generator for a cyclic code of length n if and only if g(x) divides 1 + xn.

Carollary 4.2.18 The generator polynomial g(x) for the smallest cyclic code of length n containing the word v (polynomial v(x)) is the greatest common divisor of v(x) and 1 + xn (that is, g(x) = gcd(v(x), 1 + xn)).

Proof: If g(x) is the generator polynomial then g(x) divides both v(x) and 1 + xn. But g(x) is in <{v(x), xv(x), ..., x n–1 v(x)}>, thus we have

g(x) = a(x)v(x) mod 1 + xn

or equivalently by the Division Algorithm:

g(x) = a(x)v(x) + b(x)(1 + xn).

Thus any common divisor of v(x) and 1 + xn must divide g(x) and thus g(x) is the greatest common divisor.

Example 4.2.19 Let n = 8 and v = 11011000, i.e., v(x) = 1 + x + x 3 + x 4. The gcd of v(x) and 1 + x 8 is 1 + x 2. Thus g(x) = 1 + x 2 and the smallest linear cyclic code containing v(x) has dimension of 6 and g(x) as the generator polynomial.

The Euclidean Algorithm for computing the gcd of two polynomials is discussed in the Appendix A. An alternate approach to finding the generator polynomial for a cyclic code of length n and dimension n – k involves simple row reduction. If one takes a basis (or generator matrix) and puts it into RREF with the last k columns as the ''leading columns" then the row (codeword) of minimum degree will be the generator polynomial.
< previous page | page_100 | next page >


page_101

< previous page | page_101 | next page >

Page 101

Exercises

4.2.20 For each of the words below, find the generator polynomial for the smallest linear cyclic code containing that word.

(a) 010101

(b) 010010

(c) 01100110

(d) 0101100

(e) 001000101110000

(f) 000010010000000

(g) 010111010000000

4.2.21 Find the generator polynomial of the smallest linear cycle code containing each of the following words.

(a) 101010

(b) 1100

(c) 10001000

(d) 011011

(e) 10101

(f) 111111.

4.2.22 For each of the codes C = <S> with S defined below, find the generator polynomial g(x) and then represent each word in the code as a multiple of g(x).

(a) S = {010, 011, 111}

(b) S = {1010, 0101, 1111}

(c) S = {0101, 1010, 1100}

(d) S = {1000, 0100, 0010, 0001}

(e) S = {11000, 01111, 11110, 01010}

4.3 Generating and parity check matrices for cyclic codes

One can find various generator matrices for linear cyclic codes; the simplest is the matrix in which the rows are the codewords corresponding to the generator polynomial and its first k – 1 cyclic shifts (see Theorem 4.2.13):

Example 4.3.1 Let C = {0000, 1010, 0101, 1111} be a linear cyclic code. The generator polynomial for C is g(x) = 1 + x 2. Here n = 4 and k = 2, so a basis for C consists of

g(x) = 1 + x 2 ↔ 1010, xg(x) = x + x 3 ↔ 0101,

as can be easily verified. A generating matrix for C is
< previous page | page_101 | next page >


page_102

< previous page | page_102 | next page >

Page 102

Example 4.3.2 Let C be the linear cyclic code of length n = 7 with generator polynomial g(x) = 1 + x + x 3 of degree n – k = 3. Then k = 4, so a basis for C is

and a generating matrix for C is

Let C be a linear cyclic code of length n and dimension k (so the generator polynomial g(x) has degree n – k). The k information digits (a 0, a 1, ..., a k–1) to be encoded can be represented by the polynomial a(x) = a 0 + a 1 x + ... + a k–1 x k–1 called the information or message polynomial. Encoding consists simply of polynomial multiplication; that is, a(x) is encoded as a(x)g(x) = c(x). So instead of storing the entire k × n generator matrix one only has to store the generator polynomial, which is a significant improvement in terms of the complexity of encoding.

The inverse operation to polynomial multiplication is polynomial division. Hence finding the message corresponding to the closest codeword c(x) to the received word consists of dividing c(x) by g(x), thus recovering the message polynomial a(x).

Example 4.3.3 Let g(x) = 1 + x + x 3 and n = 7. Then k = 7 – 3 = 4. Let a(x) = 1 + x 2 be the message polynomial corresponding to the word a = 1010. The message a(x) is encoded as c(x) = a(x)g(x), so

c(x) = (1 + x 2)(1 + x + x 3) = 1 + x + x 2 + x 5

with c = 1110010 as the corresponding codeword.

If c(x) = 1 + x + x 4 + x 6 then the corresponding message polynomial is c(x)/g(x) = a(x) = 1 + x 3 corresponding to the message a = 1001.

Exercises

4.3.4 Let g(x) = 1 + x 2 + x 3 be the generator polynomial of a linear cyclic code of length 7.

(a) Encode the following message polynomials: 1 + x 3, x, x + x 2 + x 3.
< previous page | page_102 | next page >


page_103

< previous page | page_103 | next page >

Page 103

(b) Find the message polynomial corresponding to the codewords c(x): x 2 + x 4 + x 5, 1 + x + x 2 + x 4, x 2 + x 3 + x 4 + x 6.

4.3.5 Find a basis and generating matrix for the linear cyclic code of length n with generator polynomial g(x).

(a) n = 7, g(x) = 1 + x 2 + x 3

(b) n = 9, g(x) = 1 + x 3 + x 6

(c) n = 15, g(x) = 1 + x + x 4

(d) n = 15, g(x) = 1 + x 4 + x 6 + x 7 + x 8

(e) n = 15, g(x) = 1 + x + x 2 + x 4 + x 5 + x 8 + x 10.

4.3.6 Show that the linear code with given generator matrix is cyclic and find the generator polynomial.

Having found an effective generating matrix for a linear cyclic code described in terms of its generator polynomial, we next turn our attention to finding a parity check matrix for such codes. To do so, we find a matrix H such that wH = 0 if and only if w is a codeword. So we begin by writing w(x) = c(x) + e(x), thinking of c(x) as a codeword and e(x) as the error polynomial.

The syndrome polynomial, s(x), is defined by s(x) = w(x) mod g(x). Assuming g(x) has degree n – k, then s(x) will have degree less than n – k and will correspond to a binary word s, of length n – k. Since w(x) = c(x) + e(x) and c(x) = a(x)g(x) for some polynomial a(x), we have that s(x) = e(x) mod g(x). That is, the syndrome polynomial is dependent only on the error.

We can define a matrix H in which the ith row ri is the word of length n – k corresponding to ri(x) = x i mod g(x). It turns out that this matrix is a parity check matrix for the code. For, if w is a received word then w(x) = c(x) + e(x), so

Then s(x) = 0 if and only if w(x) is a codeword, so H is a parity check matrix. Also, if wH = s then s corresponds to s(x) = w(x) mod g(x). It is now clear why we refer to s(x) as the syndrome polynomial. We will use this representation of the syndrome when we consider the correction of bursts of errors in Chapter 7.
< previous page | page_103 | next page >


page_104

< previous page | page_104 | next page >

Page 104

Example 4.3.7 Let n = 7, and g(x) = 1 + x + x 3. Then n – k = 3. We produce H as follows.

If w(x) = 1 + x 5 + x 6 is received, w = 1000011, then wH = s = 110 and s(x) = 1 + x = 1 + x 5 + x 6 mod (1 + x + x 3) = w(x) mod g(x).

Exercises

4.3.8 Find a parity check matrix for the linear cyclic code of length 7 with generator g(x) = 1 + x + x 2 + x 4.

4.3.9 Find a parity check matrix for a cyclic code of length n and with generator g(x):

(a) n = 6, g(x) = 1 + x 2

(b) n = 6, g(x) = 1 + x 3

(c) n = 8, g(x) = 1 + x 2

(d) n = 9, g(x) = 1 + x 3 + x 6

(e) n = 15, g(x) = 1 + x + x 4 (this generates a Hamming code)

(f) n = 23, g(x) = 1 + x + x 5 + x 6 + x 7 + x 9 + x 11 (this generates a Golay code)

(g) n = 15, g(x) = 1 + x 4 + x 6 + x 7 + x 8 (this generates a 2-error correcting BCH code, constructed in Chapter 5).

4.4 Finding cyclic codes

To construct a linear cyclic code of length n and dimension k, one must find a factor of 1 + xn having degree n – k. Of course there maybe several choices or none for given n and k. There is also the question of minimum distance for cyclic

< previous page | page_104 | next page >


page_105

< previous page | page_105 | next page >

Page 105

codes which we have not considered, a question which is not settled in general. We will put this issue off until later.

To reiterate, the fact that every generator must divide 1 + xn enables us to find all linear cyclic codes of a given length n. All we have to do is find all factors of 1 + xn, which means first finding all irreducible factors.

A polynomial f(x) in K[x] of degree at least one is irreducible if it is not the product of two polynomials in K[x], both of which have degree at least one. Finding the irreducible factors (which essentially gives all the factors of 1 + xn) is not all that easy. The factorization of 1 + xn for n ≤ 31 into irreducible polynomials is in Appendix B, and a method to factor 1 + xn is discussed below (see 4.4.14).

The factor 1 of 1 + xn has degree 0 and hence generates a cyclic code of dimension n; this code must be Kn, which proves that Kn is cyclic. We can also, as a special case, define the code {0} consisting of only the zero word of length n to be cyclic with "generator" g(x) = 0 = 1 + xn mod 1 + xn.

We will call these linear cyclic codes Kn and {0}, improper cyclic codes. Otherwise, the code is a proper cyclic code.

Example 4.4.1 For n = 3, 1 + x 3 = (1 + x)(1 + x + x 2) is the factorization of 1 + x 3 into irreducible factors. Thus there are two proper cyclic codes of length 3. One has generator g(x) = 1 + x and generating matrix

The code is C = {000, 110, 011, 101}. The other code has generator g(x) = 1 + x + x 2 and generating matrix G = [111], so is the code C = {000, 111}.

Example 4.4.2 For n = 6, we factor 1 + x 6 into irreducible factors.

1 + x 6 = (1 + x 3)2 = (1 + x)2(1 + x + x 2)2.

Then to find the generators of proper linear cyclic codes of length 6, we form all possible products of these factors except for 1 and 1 + x 6. Each such product is the generator for a proper cyclic linear code of length 6. These products and the dimension of the cyclic linear code of length 6 that each product generates are given in the following table.

generator | dimension
1 + x | 5
(1 + x)2 = 1 + x 2 | 4
1 + x + x 2 | 4
(1 + x + x 2)2 = 1 + x 2 + x 4 | 2
(1 + x)(1 + x + x 2) = 1 + x 3 | 3
(1 + x)2(1 + x + x 2) = 1 + x + x 3 + x 4 | 2
(1 + x)(1 + x + x 2)2 = 1 + x + x 2 + x 3 + x 4 + x 5 | 1
< previous page | page_105 | next page >


page_106

< previous page | page_106 | next page >

Page 106

Theorem 4.4.3 If n = 2r s then 1 + xn = (1 + xs)2r.

Proof: If n = 2s, then (1 + xs)2 = 1 + xs + xs + x 2s= 1 + x 2s. We then proceed by induction on r.

Corollary 4.4.4 Let n = 2rs, where s is odd and let 1 + xs be the product of z irreducible polynomials. Then there are (2r + 1)z linear cyclic codes of length n and (2r + 1)z – 2 proper linear cyclic codes of length n.

Example 4.4.5 In Example 4.4.1 it is shown that 1 + x 3 is the product of two irreducible polynomials, namely 1 + x and 1 + x + x 2. By applying Corollary 4.4.4 with r = 0, s = 3 and z = 2 we find that there are (20 + 1)2 = 4 linear cyclic codes of length 3, 2 of which are proper (as shown in Example 4.4.1). Also, for 1 + x 6, we have n = 6 = 213 so r = 1, z is still 2 thus there are (2 + 1)2 = 9 linear cyclic codes of length 6, 7 of which are proper (as was shown in Example 4.4.2).

Exercises

4.4.6 Find the number of proper linear cyclic codes of length n, where

(a) n = 4,

(b) n = 5,

(c) n = 7,

(d) n = 14,

(e) n = 56,

(f) n = 15,

(g) n = 120,

(h) n = 1024.

4.4.7 Find the generator polynomial for all proper linear cyclic codes of length n, where (a) n = 4 and (b) n = 5.

4.4.8 Find two generators of degree 4 for a linear cyclic code of length 7.

4.4.9 Find a generator and a generating matrix for a linear code of length n and dimension k where

(a) n = 12, k = 5

(b) n = 12, k = 7

(c) n = 14, k = 5

(d) n = 14, k = 6

(e) n = 14, k = 8.

4.4.10 Show that the Golay Code C 23 is equivalent to a linear cyclic code.

One can find all cyclic codes or equivalently factor 1 + xn, by a relatively simple procedure. Throughout our discussion we will assume that n is odd.

The first step involves generating all polynomials I(x) (mod 1 + xn) such that I(x) = I(x)2 (mod 1 + xn). These polynomials are called idempotent polynomials. It is easy to see that if u(x) and v(x) are idempotent, so is their sum u(x) + v(x) and product u(x)v(x) (mod 1 + xn). Thus we need to construct only a ''basic" set of idempotent polynomials. To do this we need to partition Zn = {0, 1, ..., n – 1} into "classes."

Let Ci = {s = 2j · i(mod n) | j = 0, 1, ..., r} where 1 = 2r mod n.
< previous page | page_106 | next page >


page_107

< previous page | page_107 | next page >
Page 107 Example 4.4.11 For n = 7 we have C 0 = {0}, C 1 = {1, 2, 4} = C 2 = C 4, and C 3 = {3, 5, 6} = C 5 = C 7. For n = 9 we have C 0 = {0}, C 1 = {1, 2, 4, 8, 7, 5}, and C 3 = {3, 6}. Next for each different class Ci we form a polynomial We will show that ci(x) is an idempotent, and note in passing that any idempotent I(x) (mod 1 + xn) is To see that ci(x) is an idempotent, since if j ∈ Ci then so is 2j (mod n). Example 4.4.12 For n = 7 we have, Then any idempotent polynomial (mod 1 + x 7) can be expressed as I(x) = a 0 c 0(x) + a 1 c 1(x) + a 3 c 3(x), ai ∈{0, 1}. Thus we have 23 – 1 different idempotents (mod 1 + x 7). (We ignore I(x) = 0 which is trivially idempotent.) The connection between idempotents and cyclic codes is the following: Theorem 4.4.13 Every cyclic code contains a unique idempotent polynomial which generates the code.
< previous page | page_107 | next page >


page_108

< previous page | page_108 | next page >

Page 108

Proof: Let g(x) be the generator of a cyclic code of length n and let g(x)h(x) = 1 + xn (n is odd). Then gcd(h(x), g(x)) = 1 and by the Euclidean Algorithm (Appendix A) there exists polynomials t(x), s(x) such that

1 = t(x)g(x) + s(x)h(x).

Multiplying both sides by t(x)g(x) gives,

t(x)g(x) = (t(x)g(x))2 + t(x)s(x)(1 + xn)

or

t(x)g(x) = (t(x)g(x))2 mod 1 + xn.

Thus t(x)g(x) is an idempotent and

g(x) = gcd(t(x)g(x), 1 + xn).

Example 4.4.14 To find all cyclic codes of length 9, we simply find all idempotent polynomials and find the corresponding generator polynomial. Since

C 0 = {0}, C 1 = {1, 2, 4, 8, 7, 5}, C 3 = {3, 6}

we have

c 0(x) = 1, c 1(x) = x + x 2 + x 4 + x 5 + x 7 + x 8, c 3(x) = x 3 + x 6,

and

I(x) = a 0 c 0(x) + a 1 c 1(x) + a 3 c 3(x).

Idempotent polynomial I(x) | The generator polynomial g(x) = gcd(I(x), 1 + x 9)
1 | 1
x + x 2 + x 4 + x 5 + x 7 + x 8 | 1 + x + x 3 + x 4 + x 6 + x 7
x 3 + x 6 | 1 + x 3
1 + x + x 2 + x 4 + x 5 + x 7 + x 8 | 1 + x + x 2
1 + x 3 + x 6 | 1 + x 3 + x 6
x + x 2 + x 3 + x 4 + x 5 + x 6 + x 7 + x 8 | 1 + x
1 + x + x 2 + x 3 + x 5 + x 6 + x 7 + x 8 | 1 + x + x 2 + x 3 + x 5 + x 6 + x 7 + x 8

Exercises

4.4.15 Find all idempotents polynomials mod 1 + xn, and the corresponding generator polynomials for,

(a) n = 5

(b) n = 7

(c) n = 11

(d) n = 15

(e) n = 31
< previous page | page_108 | next page >


page_109

< previous page | page_109 | next page >

Page 109

4.5 Dual cyclic codes

Another fact about cyclic codes which is useful, is that the dual codes are also cyclic. We will in fact give a procedure for constructing the generator polynomial of the dual code.

It is a simple matter to see that the dual of a cyclic code is cyclic. This follows directly from the fact that if a · b = 0 then π(a) · π(b) = 0 where π is the cyclic shift, as the following argument shows. (Note that a · b = a 0 b 0 + a 1 b 1 + · · · + an bn and π(a) · π(b) = a 1 b 1 + a 2 b 2 + ··· + an bn + a 0 b 0 = a · b = 0.) Consider the cyclic code which is generated by the word v; so C = <{v, π(v), ..., π n–1(v)}>. If u ∈ C⊥ then π i(v) · u = 0 for i = 0, 1, ..., n – 1. However this means that π i + 1(v) · π(u) = 0 and thus π(u) is orthogonal to <{π(v), π 2(v), ..., π n(v)}> = C because π n(v) = v. Since u ∈ C⊥ implies π(u) ∈ C⊥ we conclude that C⊥ is cyclic.

To find the generator of the dual we need to relate the product of polynomials and the dot product of vectors.

Lemma 4.5.1 Let a ↔ a(x), b ↔ b(x) and b′ ↔ b′(x) = xn b(x–1) mod 1 + xn, then a(x)b(x) mod 1 + xn = 0 if and only if π k(a) · b′ = 0 for k = 0, 1, ..., n – 1.

Proof: Let c(x) = a(x)b(x) mod 1 + xn. Then the coefficient of xk in c(x) is

ck = a k b 0 + a k + 1 b n–1 + ··· + a n–1 b k + 1 + a 0 b k + ··· + a k–1 b 1

since xk ≡ x n + k (mod 1 + x n). Note that if a = (a 0, a 1, ..., a n–1) and b = (b 0, b 1, ..., b n–1) then b′ = (b 0, b n–1, b n–2..., b 1) and so ck = π k(a)·b′. Thus ck = 0 for k = 0, 1, ..., n – 1 if and only if c(x) = 0 = a(x)b(x) mod 1 + xn.

Again let C be a cyclic linear code of length n and g(x) be the generator polynomial for C. We know that g(x) divides 1 + xn and thus there is a unique polynomial h(x), such that 1 + xn = g(x)h(x). By Lemma 4.5.1 we know that xn h(x–1) is in C⊥, but we want to find the generator for C⊥.

Theorem 4.5.2 If C is a linear cyclic code of length n and dimension k with generator g(x) and if 1 + xn = g(x)h(x) then C⊥ is a cyclic code of dimension n – k with generator xk h(x–1).

Proof: Since C has dimension k, g(x) has degree n – k and thus h(x) has degree k. Since

g(x)h(x) = 1 + xn

we have

g(x–1)h(x–1) = 1 + (x–1)n

and
< previous page | page_109 | next page >


page_11

< previous page | page_11 | next page >

Page 11

4. for some v′ ∈ Kn, v+v′ = v′+v = 0

5. v+w = w+v

6. av ∈ Kn

7. a(v+w) = av+aw

8. (a+b)v = a(bv)

9. (ab)v = a(bv)

10. 1v = v.

Exercises

1.7.1 Show that if v is a word in Kn then v+v = 0.

1.7.2 Show that if v and w are words in Kn and v+w = 0 then v = w.

1.7.3 Show that if u, v and w are words in Kn and u+v = w then u+w = v.

Notice that if v is sent over a BSC and w is received then 0 occurs in a component of v+w if the corresponding component of v was correctly transmitted and a 1 occurs if the component was incorrectly transmitted. v+w is called the error pattern, or the error. For example if v = 10101 is transmitted and w = 01100 is received then errors occurred in the 1st, 2nd and 5th components. The error pattern is v+w = 11001.

1.8 Weight and distance

We introduce two important terms. Let v be a word of length n. The Hamming weight, or simply the weight, of v is the number of times the digit 1 occurs in v. We denote the weight of v by wt(v). For example, wt(110101) = 4 and wt(00000) = 0.

Let v and w be words of length n. The Hamming distance, or simply distance, between v and w is the number of positions in which v and w disagree. We denote the distance between v and w by d(v,w). For example, d(01011, 00111) = 2 and d(10110, 10110) = 0.

Note that the distance between v and w is the same as the weight of the error pattern u = v+w:

d(v,w) = wt(v+w).

For example, if v = 11010 and w = 01101, we have d(v,w) = d(11010, 01101) = 4, and wt(v+w) = wt(11010+01101) = wt(10111) = 4. Thus the probability formula in Section 1.6 can be re-expressed as

φp(v,w) = p n−wt(u)(1 − p)wt(u),

where u is the error pattern u = v+w. We refer to [image] as the probability of the error pattern u = v+w.
< previous page | page_11 | next page >


page_110

< previous page | page_110 | next page >

Page 110

Thus xk h(x–1) is a factor of 1 + xn, having degree k and hence the generator polynomial for the linear cyclic code, C⊥ of dimension n – k containing xn h(x–1).

Example 4.5.3 g(x) = 1 + x + x 3 is the generator of a cyclic code of length 7 and dimension k = 7 – 3 = 4. Since g(x) is a factor of 1 + x 7 we can find h(x) where 1 + x 7 = g(x)h(x) by long division. In this case h(x) = 1 + x + x 2 + x 4. The generator for C⊥ is g⊥(x) = x 4 h(x–1) = x 4(1 + x–1 + x–2 + x–4) = 1 + x 2 + x 3 + x 4 which corresponds to 1011100 = w. Clearly g · w = (11010000) · (1011100) = 0 and πk(g) · w = 0 as well. Note that g⊥(x) ≠ h(x).

Example 4.5.4 Let g(x) = 1 + x + x 2 be the generator for a linear cyclic code of length 6. We find h(x) = 1 + x + x 3 + x 4 satisfies g(x)h(x) = 1 + x 6. Therefore g⊥(x) = x 4 h(x–1) = x 4(1 + x–1 + x–3 + x–4) = x 4 + x 3 + x + 1 is the generator for the dual code. Note in this example g⊥(x) = h(x).

Exercises

4.5.5 Find the generator polynomial for the dual code of the cyclic code of length n having generator polynomial g(x) where:

(a) n = 6, g(x) = 1 + x 2

(b) n = 6, g(x) = 1 + x 3

(c) n = 8, g(x) = 1 + x 2

(d) n = 9, g(x) = 1 + x 3 + x 6

(e) n = 15, g(x) = 1 + x + x 4

(f) n = 15, g(x) = 1 + x 4 + x 6 + x 7 + x 8

(g) n = 23, g(x) = 1 + x + x 5 + x 6 + x 7 + x 9 + x 11

(h) n = 7, g(x) = 1 + x + x 2 + x 4
< previous page | page_110 | next page >


page_111

< previous page | page_111 | next page >

Page 111

Chapter 5
BCH Codes

5.1 Finite fields

In this chapter we consider a special class of cyclic codes and a different approach to decoding them, one which utilizes Galois fields GF(2r).

Recall that a polynomial d(x) is a divisor or factor of f(x) if f(x) = g(x)d(x). Of course 1 and f(x) are always divisors of f(x) but these are trivial. Any other divisor is said to be a nontrivial or proper divisor of f(x). A polynomial f(x) ∈ K[x] is said to be irreducible over K if it has no proper divisors in K[x]; otherwise it is said to be reducible (or factorable) over K.

Example 5.1.1 Polynomials x and 1 + x are irreducible by definition; 1 + x + x 2 has neither x nor 1 + x as a divisor so it too is irreducible. However, x 2, 1 + x 2, and x + x 2 are not irreducible: x 2 and x + x 2 both have x as a divisor; 1 + x 2 has 1 + x as a divisor.

In general 1 + x is a divisor or factor of f(x) if and only if 1 is a root of f(x); that is, if and only if f(1) = 0. Note that 1 + x is a factor of f(x) = 1 + x 2 and f(1) = 1 + 1 = 0. Similarly x is a factor of g(x) if and only if g(0) = 0. However finding other irreducible factors of a polynomial is more difficult and at this point is simply a matter of trial and error.

Example 5.1.2 If f(x) = 1 + x + x 2 + x 3, then f(1) = 1 + 1 + 1 + 1 = 0, and so 1 + x is a factor of f(x). By long division f(x) = (1 + x)(1 + x 2) = (1 + x)3. On the other hand, if g(x) = 1 + x + x 3, then g(0) = 1 ≠ 0 and g(1) = 1 ≠ 0, so g(x) has no linear factor. Therefore g(x) is irreducible over K, since if a cubic polynomial is reducible then it must have a linear factor.

Example 5.1.3 Let f(x) = 1 + x + x 4. Since f(0) ≠ 0 and f(1) ≠ 0, f(x) has no linear factors. So, if f(x) is reducible, then f(x) must have an irreducible quadratic factor. The only irreducible quadratic over K is g(x) = 1 + x + x 2. After dividing g(x) into f(x), we find a non-zero remainder. So 1 + x + x 2 is not a factor of f(x). Therefore f(x) is irreducible over K.
< previous page | page_111 | next page >


page_112

< previous page | page_112 | next page >

Page 112

Exercises

5.1.4 Determine whether each of the following polynomials is irreducible over K.

(a) f(x) = 1 + x 2 + x 4

(b) f(x) = 1 + x 8

(c) f(x) = 1 + x 2 + x 3 + x 5

(d) f(x) = 1 + x 2 + x 6

(e) f(x) = 1 + x 4 + x 5

(f) f(x) = 1 + x + x 3 + x 7

5.1.5 Find all irreducible polynomials of degree 3 and 4 over K.

5.1.6 Find all irreducible polynomials of degree 5 over K.

An irreducible polynomial over K of degree n > 1 is said to be primitive if it is not a divisor of 1 + xm for any m < 2n – 1. We will see that an irreducible polynomial of degree n always divides 1 + xm when m = 2n – 1.

Example 5.1.7 Since 1 + x + x 2 is not a factor of 1 + xm for m < 3 = 22 – 1, it is primitive. Similarly 1 + x + x 3 is not a factor of 1 + xm for any m < 7 = 23 – 1 and thus it too is primitive.

However 1 + x 5 = (1 + x)(1 + x + x 2 + x 3 + x 4) and 1 + x + x 2 + x 3 + x 4 is irreducible (see Exercise 5.1.5) but 5 < 15 = 24 – 1 and thus 1 + x + x 2 + x 3 + x 4 is not primitive.

Recall that we can define addition and multiplication of polynomials modulo a polynomial h(x) of degree n. Let Kn[x] denotes the set of all polynomials in K[x] having degree less than n. Of course each word in Kn corresponds to a polynomial in Kn[x] so we can in effect define addition and multiplication of words in Kn.

In this chapter we introduce the additional structure of finite fields to assist in constructing and decoding codes. We already have a definition of addition and multiplication of words in Kn, but for this to form a field we need to be careful in our choice of h(x). For example, in a field it must be the case that if ab = 0 then either a = 0 or b = 0.

Example 5.1.8 We try using multiplication of polynomials modulo 1 + x 4 to define multiplication of words in K 4. However,

so (0101)(0101) = 0000, but 0101 ≠ 0000 in K 4. Thus K 4 cannot be a field under this definition of multiplication.
< previous page | page_112 | next page >


page_113

< previous page | page_113 | next page >

Page 113

The difficulty in the last example arises because 1 + x 4 is not irreducible over K. The way to define multiplication in Kn in order to make Kn into a field is to define multiplication in Kn modulo an irreducible polynomial of degree n. We leave the proof that this is the field GF(2n) to a course in modern algebra.

Example 5.1.9 Define multiplication in K 4 using the irreducible polynomial h(x) = 1 + x + x 4. To find the product (1101)(0101) note that

(1101)(0101) ↔ (1 + x + x 3)(x + x 3).

But (1 + x + x 3)(x + x 3) = x + x 2 + x 3 + x 6 and

x = x + x 2 + x 3 + x 6 mod (1 + x + x 4).

Thus (1101)(0101) = 0100 ↔ x.

Exercises

5.1.10 Define multiplication in K 4 modulo h(x) = 1 + x + x 4. Calculate the following products.

(a) (0011)(1011)

(b) (1110)(1001)

(c) (1010)(0110)

(d) (0100)(0010)

(e) (1100)(0111)

(f) (1111)(0001)

5.1.11 Find all products of elements in K 2 using 1 + x + x 2 to define multiplication (that is, make a multiplication table).

Example 5.1.12 Let us consider the construction of GF(23) using the primitive polynomial h(x) = 1 + x + x 3 to define multiplication. We do this by computing x i mod h(x):

word | ↔ | xi mod h(x)
100 |  | 1
010 |  | x
001 |  | x 2
110 |  | x 3 ≡ 1 + x
011 |  | x 4 ≡ x + x 2
111 |  | x 5 ≡ 1 + x + x 2
101 |  | x 6 ≡ 1 + x 2

To compute (110)(001) ↔ (1 + x)x 2 note that from the above table 1 + x = x 3 mod h(x) so

thus (110)(001) = 111.
< previous page | page_113 | next page >


page_114

< previous page | page_114 | next page >

Page 114

Using a primitive polynomial to construct GF(2r) makes computing in the field much easier than using a non-primitive irreducible polynomial. To see this, let β ∈ Kn represent the word corresponding to x mod h(x), where h(x) where h(x) is a primitive polynomial of degree n. Then β i ↔ x i mod h(x). Note that 1 = xm mod h(x) means that 0 = 1 + xm mod h(x) and thus that h(x) divides 1 + xm. Since h(x) is primitive we know that h(x) does not divide 1 + xm for m < 2n – 1 and thus β m ≠ 1 for m < 2n – 1. Since β j = β i for j ≠ i if and only if βi = βj–i β i which implies β j–i = 1, we conclude that

Kn\{0} = {βi | i = 0, 1, ..., 2n – 2}.

That is, every non-zero word in Kn can be represented by some power of β This is the property that makes multiplication in the field easy.

word | polynomial in x (modulo h(x)) | power of β
0000 | 0 | —
1000 | 1 | β 0 = 1
0100 | x | β
0010 | x 2 | β 2
0001 | x 3 | β 3
1100 | 1 + x ≡ x 4 | β 4
0110 | x + x 2 ≡ x 5 | β 5
0011 | x 2 + x 3 ≡ x 6 | β 6
1101 | 1 + x + x 3 ≡ x 7 | β 7
1010 | 1 + x 2 ≡ x 8 | β 8
0101 | x + x 3 ≡ x 9 | β 9
1110 | 1 + x + x 2 ≡ x 10 | β 10
0111 | x + x 2 + x 3 ≡ x 11 | β 11
1111 | 1 + x + x 2 + x 3 ≡ x 12 | β 12
1011 | 1 + x 2 + x 3 ≡ x 13 | β 13
1001 | 1 + x 3 ≡ x 14 | β 14
Table 5.1: Construction of GF(24) using h(x) = 1 + x + x 4.

An element α ∈ GF(2r) is primitive if α m ≠ 1 for 1 ≤ m < 2r – 1. Equivalently, α is primitive if every non-zero word in GF(2r) can be expressed as a power of α. From the above discussion we see that if a primitive polynomial is used to construct GF(2r), with β being the word defined above, then β is a primitive element.

Example 5.1.13 Construct GF(24) using the primitive polynomial h(x) = 1 + x + x 4. Write every vector as a power of β ↔ x mod h(x) (see Table 5.1). Note that β 15 = 1.

To compute (0110)(1101) = β 5 · β 7 = β 12 = 1111 since (x + x 2)(1 + x + x 3) ≡ x 5 · x 7 ≡ x 12 (mod h(x)).
< previous page | page_114 | next page >


page_115

< previous page | page_115 | next page >

Page 115

Exercises

5.1.14 Use GF(24) constructed in Table 5.1 to compute the products in K 4 in Exercise 5.1.10.

5.1.15 Construct the following fields as in Example 5.1.13 (Table 5.1).

(a) Construct GF(22).

(b) Construct GF(23) using h(x) = 1 + x 2 + x 3.

(c) Construct GF(24) using h(x) = 1 + x 3 + x 4.

(d) Construct GF(25) using h(x) = 1 + x 2 + x 5.

5.1.16 Show that if h(x) ∈ K[x] is an irreducible polynomial of degree n, then h(x) divides 1 + xm for some m ≤ 2n – 1.

5.1.17 Find all primitive elements in GF(24) (see Table 5.1).

5.1.18 Show that β i ∈ GF(2r) is primitive iff gcd(i, 2r – 1) = 1.

5.2 Minimal polynomials

Recall that α, an element in a field F = GF(2r) is said to be a root of a polynomial p(x) ∈ F[x] if and only if p(α) = 0. That is, if p(x) = a 0 + a 1 x + · · · + ak xk then

p(α) = a 0 + a 1 α + · · · + akαk = 0.

Example 5.2.1 Let p(x) = 1 + x 3 + x 4, and let β be the primitive element in GF(24) constructed using h(x) = 1 + x + x 4 (see Table 5.1).

So β is not a root of p(x). However

Since p(β7) = 0, β 7 is a root of p(x). Note that we used the convention that 1 ↔ 1000 and 0 ↔ 0000 as well as the fact that β 15 = 1. Thus β 21 = β 15 β 6 = 1 · β 6 = β 6 and β 28 = β 15 β 13 = 1 · β 13 = β 13.

In general the order of non-zero element α in GF(2r) is the smallest positive integer m such that α m = 1. We know that for any non-zero α in GF(2r), α has

< previous page | page_115 | next page >


page_116

< previous page | page_116 | next page >

Page 116

order m≤2r–1. In particular, α in GF(2r) is a primitive element if it has order 2r – 1.

For any element α in GF(2r), we define the minimal polynomial of α as the polynomial in K[x] of smallest degree having α as a root. Let mα(x) denote the minimal polynomial of α. Note that if α has order m (that is, α m = 1) then α is a root of 1 + xm, so every element in GF(2r), is a root of some polynomial in K[x].

To find the minimal polynomial of an element of GF(2r), it will help to have some facts concerning minimal polynomials.

Theorem 5.2.2 Let α ≠ 0 be an element of GF(2r). Let m α(x) be the minimal polynomial of α. Then

(a) mα(x) is irreducible over K,

(b) if f(x) is any polynomial over K such that f(α) = 0, then mα(x) is a factor of f(x),

(c) the minimal polynomial is unique, and

(d) the minimal polynomial mα(x) is a factor of [image].

Proof: (a) If mα(x) = g(x)h(x), then mα(α) = 0 implies g(α)h(α) = 0. Thus either g(α) = 0 or h(α) = 0. Since mα(x) is the polynomial of smallest degree such that mα(x) = 0, then either g(x) = 1 or h(x) = 1. Therefore mα(x) is irreducible over K.

(b) By the Division Algorithm,

f(x) = mα(x)g(x) + r(x),

where r(x) = 0 or deg r(x) < deg mα(x). Now f(α) = 0, so since

f(α) = mα(α)g(α) + r(α) = 0 · g(α) + r(α) = r(α)

we have that r(α) = 0. By the minimality of the degree of mα(x), r(x) = 0. Therefore f(x) = mα(x)q(x), and mα(x) is a factor of f(x).

(c) If m′(x) is also a polynomial of smallest degree such that m′(α) = 0, then, by part (b), mα(x) is a factor of m′(x) and m′(x) is a factor of mα(x). Therefore mα(x) = m′(x), so the minimal polynomial is unique.

(d) Let β be a primitive element in GF(2r) and α = β i. Then [image]. Thus α is a root of 1 + x 2r–1 and by (b) mα(x). is a factor of [image].

Finding the minimal polynomial of α, α ∈ GF(2r), reduces to finding a linear combination of the vectors {1, α, α 2, ..., α r} which sums to 0. Since any set of r + 1 vectors in Kr is dependent we know such a combination does exist.

Once we have constructed GF(2r) using a primitive polynomial, it is naturally convenient to represent mα(x) by mi(x) where α = β i. We introduce this notation in the following example.
< previous page | page_116 | next page >


page_117

< previous page | page_117 | next page >

Page 117

Example 5.2.3 Find the minimal polynomial of α = β 3, α ∈ GF(24) constructed using h(x) = 1 + x + x 4 (see Table 5.1). Let mα(x) = m 3(x) = a 0 + a 1 x + a 2 x 2 + a 3 x 3 + a 4 x 4 then we must find the values for a 0, a 1, ..., a 4 ∈ {0, 1}. Note,

Solving for a 0, a 1, a 2, a 3, a 4 we find that

a 0 = a 1 = a 2 = a 3 = a 4 = 1 and

mα(x) = 1 + x + x 2 + x 3 + x 4.

The roots of mα(x) are {α, α 2, α 4, α 8} = {β3, β 6, β 12, β 9}, and thus m 3(x) = m 6(x) = m 9(x) = m 12(x) (where mi(x) denotes the minimal polynomial of β i).

If the minimal polynomials for all elements in GF(2r) are being sought then we have other useful facts. Recall that f(x)2 = f(x 2), so

This follows from the fact that (a + b)2 = a 2 + b 2 and the fact that [image] since a i ∈ {0, 1}.

Thus if f(α) = 0 then f(α 2) = (f(α))2 = 0 and so α 2 is also a root of f(x). Similarly f(α 4) = (f(α 2))2 = 0, etc. and so we have that if α is a root of f(x) so are [image], etc. With some more effort one can prove:

Theorem 5.2.4 Let α be an element in GF(2r) with minimal polynomial mα(x), then [image] is the set of all the roots of mα(x). In particular, the degree of mα(x) is [image].

Example 5.2.5 Let m 5(x) be the minimal polynomial of α = β 5, β 5 ∈ GF(24) (see Table 5.1). Since {α, α 2, α 4, α 8} = {β5, β 10} by Theorem 5.2.3 the roots of m 5(x) are β 5 and β 10 which means that degree (m 5(x)) = 2 (from Theorem 5.2.4). Thus m 5(x) = a 0 + a 1 x + a 2 x 2, hence

Thus a 0 = a 1 = a 2 = 1 and m 5(x) = 1 + x + x 2.

Similarly we can find the minimal polynomials of the rest of the field elements in GF(24) constructed using 1 + x + x 4. The results are summarized in Table 5.2.
< previous page | page_117 | next page >


page_118

< previous page | page_118 | next page >

Page 118

element of GF(24) | minimal polynomial
0 | x
1 | 1 + x
β, β 2, β 4, β 8 | 1 + x + x 4
β 3, β 6, β 9, β 12 | 1 + x + x 2 + x 3 + x 4
β 5, β 10 | 1 + x + x 2
β 7, β 11, β 13, β 14 | 1 + x 3 + x 4
Table 5.2: Minimal polynomials in GF(24).

Exercises

5.2.6 Verify the entries in Table 5.2, for GF(24).

5.2.7 Find the minimal polynomial of each element of GF(23) constructed using p(x) = 1 + x + x 3 (see Exercise 5.1.15).

5.2.8 Find the minimal polynomial of each element of GF(24) constructed using p(x) = 1 + x 3 + x 4 (see Exercise 5.1.15).

5.2.9 Find the minimal polynomial of each element of GF(25) constructed using p(x) = 1 + x 2 + x 5 (see Exercise 5.1.15).

5.2.10 Show that 1 + x + x 2 = (β5 + β 10 + x) (use Table 5.1).

5.2.11 Show that mα(x) is a primitive polynomial if and only if α is a primitive element.

5.3 Cyclic Hamming codes

We already know that Hamming codes have the important advantages of being perfect single-error-correcting codes and of admitting a very simple decoding scheme. In this section we show that there is a cyclic Hamming code of length n = 2r–1, for each r ≥ 2. This code has the added advantage common to all cyclic codes of easy encoding.

The parity check matrix of a Hamming code of length n = 2r – 1 has as its rows all 2r – 1 nonzero words of length r. If β is a primitive element of GF(2r), then by definition the powers of β are all distinct. Therefore we can construct a Hamming code of length n = 2r – 1 which has
< previous page | page_118 | next page >


page_119

< previous page | page_119 | next page >

Page 119

as its parity check matrix. Note H is a (2r – 1) × r matrix. Next, notice that for any received word w = w 0 w 1...w n–1, wH = w 0 β 0 + w 1 β + · · · + w n–1 β n–1 ↔ w(β), so w is a codeword if β is a root of w(x). Therefore, by Theorem 5.2.2 (b), m β(x) divides every codeword and clearly is a codeword itself, so this code is cyclic and m β(x) is its generator polynomial. So we have the following result.

Theorem 5.3.1 A primitive polynomial of degree r is the generator polynomial of a cyclic Hamming code of length 2r – 1.

Example 5.3.2 Let r = 3, so n = 23 – 1 = 7. Use p(x) = 1 + x + x 3 to construct GF(23), and β ↔ 010 as the primitive element. Recall that β i ↔ x i mod p(x). Therefore a parity check matrix for a Hamming code of length 7 is

which is the same as the parity check matrix of the cyclic code with a generator polynomial p(x) = m β(x).

Decoding the cyclic Hamming code is easy. If the generator is the primitive polynomial mα(x), and w(x) is received, then w(x) = c(x) + e(x), where c(x) is a codeword, and w(α) = e(α). But since e has weight 1, we know that e(α) = α j (where j is the position of the 1 in e, labeling the positions of e with 0, 1, ..., n – 1). Therefore, the most likely error polynomial is e(x) = x j and so c(x) = w(x) + x j.

Example 5.3.3 Suppose GF(23) was constructed using 1 + x + x 3. Then m 1(x) = 1 + x + x 3 is the generator for a cyclic Hamming code of length 7. Suppose w(x) = 1 + x + x 3 + x 6 is received. Then

Thus e(x) = x 3 and c(x) = w(x) + x 3 = 1 + x 2 + x 6.
< previous page | page_119 | next page >


page_12

< previous page | page_12 | next page >

Page 12

Exercises

1.8.1 Compute the weight of each of the following words, and the distance between each pair of them: v 1 = 1001010, v 2 = 0110101, v 3 = 0011110, and v 4 = v 2 + v 3.

1.8.2 Let u = 01011, v = 11010, w = 01100. Compare each of the following pairs of quantities:

(a) wt(v+w) and wt(v)+wt(w),

(b) d(v,w) and d(v,u)+d(u,w).

We now list a number of facts concerning weight and distance. Here u, v, and w are words of length n and a is a digit.

1. 0 ≤ wt(v) ≤ n.

2. wt(v) = 0 if and only if v = 0.

3. 0 ≤ d(v,w) ≤ n.

4. d(v,w) = 0 if and only if v = w.

5. d(v,w) = d(w,v).

6. wt(v+w) ≤ wt(v)+wt(w).

7. d(v,w) ≤ d(v,u)+d(u,w).

8. wt(av) = a·wt(v).

9. d(av,aw) = a·d(v,w).

Most of these facts are immediately clear from the definitions of weight and distance. In Exercise 1.8.2, the reader constructed examples of facts 6 and 7. To construct proofs, try using the basic relation d(v,w) = wt(v+w) and Exercises 1.7.1, 1.7.2, and 1.7.3 as necessary.

Exercises

1.8.3 Construct an example in K 5 of each of the nine rules above.

1.8.4 Prove each of the nine rules above.

These facts will be used as needed and without comment in the following sections.

1.9 Maximum likelihood decoding

We are now ready to give more precise formulations of two basic problems of coding theory. Let us assume we are at the receiving end of a BSC and we want to receive a message from the transmitter at the other end. The transmitter is, of course, one we have ourselves previously designed. In fact, the design of the transmitter is one of the basic problems.
< previous page | page_12 | next page >


page_120

< previous page | page_120 | next page >

Page 120

Exercises

5.3.4 Find a parity check matrix for a cyclic Hamming code of length 7 using GF(23) constructed with 1 + x + x 3, where the generator polynomial is m 3(x). If w(x) = x + x 2 + x 4 is received find the most likely codeword c(x).

5.3.5 Repeat Exercise 5.3.4 using GF(23) constructed with p(x) = 1 + x 2 + x 3 and generator polynomial m 1(x).

5.3.6 Repeat Exercise 5.3.4 using GF(23) constructed with p(x) = 1 + x 2 + x 3 and where the generator polynomial is m 3(x).

5.3.7 Construct a parity check matrix for a cyclic Hamming code of length 15.

5.3.8 Find the generator polynomial for a cyclic code of length 15 having roots 1, β 7, β 5 ∈ GF(24) (constructed using 1 + x + x 4). Construct a parity check matrix for this code. Show that c(x) ∈ C iff wt(c) is even.

5.3.9 Show that every codeword of a cyclic code has even weight iff 1 + x is a factor of the generator polynomial.

It is probably worth stating here that more general results follow from the observations discussed in this section. Let C be a cyclic code of length n with generator polynomial g(x). Suppose α ∈ GF(2r) is a root of g(x). Then for all c(x) ∈ C, c(α) = 0 and so by Theorem 5.2.2(b) mα(x) is a divisor of c(x). We can always write g(x) as a product of minimal polynomials of elements in GF(2r). We can use this to construct a parity check matrix and to find a decoding algorithm for C. The case where [image] is discussed in Section 5.4.

5.4 BCH codes

An important class of multiple-error-correcting codes is the class of Bose-Chaudhuri-Hocquengham codes, or BCH codes. The construction and decoding procedure for general BCH codes will be developed later. First we shall construct and decode an important example of the class, namely the family of two-error-correcting BCH codes.

BCH codes are important for two reasons. First, they admit a relatively easy decoding scheme and secondly the class of BCH codes is quite extensive. Indeed, for any positive integers r and t with t ≤ 2r – 1, there is a BCH code of length n = 2r – 1 which is t-error correcting and has dimension k ≥ n – rt.

The 2 error-correcting BCH code of length 2r – 1 is the cyclic linear code that is generated by [image], where β is a primitive element in GF(2r) and r ≥ 4. Since n = 2r – 1 and g(x) divides 1 + xn (by Theorem 5.2.2(c)) g(x) is the generator polynomial for a cyclic code.

Example 5.4.1 β is a primitive element in GF(24) constructed with p(x) = 1 + x + x 4 (see Table 5.1). We have that m 1(x) = 1 + x + x 4 and m 3(x) = 1 + x +
< previous page | page_120 | next page >


page_121

< previous page | page_121 | next page >

Page 121

x2 + x3 + x4. Therefore

g(x) = m 1(x)m 3(x) = 1 + x 4 + x 6 + x 7 + x 8

is the generator for a 2 error-correcting BCH code of length 15.

Exercises

5.4.2 2 error-correcting BCH codes are defined for r > 4. What code does g(x) = m1(x)m3(x) generate when r = 3?

5.4.3 β is a primitive element of GF(24) constructed using the irreducible polynomial p(x) = 1 + x3 + x4. Find the generator polynomial g(x) the 2-error-correcting BCH code of length 15 using this representation of GF(24); that is, find g(x) = m1(x)m3(x). (See Exercise 5.1.15)

5.4.4 Find a generator polynomial for a 2 error-correcting BCH code of length 31 constructing GF(25) with the irreducible polynomial 1 + x2 + x5 (see Exercise 5.1.15).

Lemma 5.4.5 The following matrix H is a parity-check matrix for the 2 error-correcting BCH code of length 2r – 1, where β is a primitive element in GF(2r), and the generator polynomial is g(x) = m1(x)m3(x)

Since β i is an element of GF(2r), it represents a word of length r, so H is a (2r – 1) × (2r) matrix. Also, since deg(m1(x)) = r = deg(m3(x)), the degree of g(x) = m1(x)m3(x) is 2r and thus the code has dimension n – 2r = 2r – 1 – 2r. (We leave the proof that m3(x) has degree r to Exercises 5.4.9).

For example, we use GF(24) constructed in Table 5.1 with the primitive polynomial p(x) = 1 + x + x 4 to construct a 2 error-correcting BCH code C 15. We define C 15 to be the linear code with the 15 × 8 parity check matrix H, and generator polynomial m1(x)m3(x) (see Table 5.3).

Theorem 5.4.6 For any integer r ≥ 4 there is a 2 error-correcting BCH code of length n = 2r – 1, dimension k = 2r – 2r – 1 and distance d = 5 having generator polynomial m1(x)m3(x).
< previous page | page_121 | next page >


page_122

< previous page | page_122 | next page >

Page 122

Table 5.3: The parity check matrix of C 15.

For the proof that the distance is 5, we will show that it can correct 2 errors and thus has distance at least 5. From the definition of the parity check matrix it is clear that n = 2r – 1, and since m 1(x) and m 3(x) each have degree r, degree (g(x)) = n – k = 2r and so k = 2r – 2r – 1.

Exercises

5.4.7 Show that the columns of the parity check matrix of C 15 in Table 5.3 are linearly independent and hence that C 15 has dimension k = 7.

5.4.8 Show that d = 5 for C 15 by using the parity check matrix.

5.4.9 Show that if β is a primitive element of GF(2r), r > 2 then [image] and [image]. Therefore m 1(x) and m 3(x) both have degree r.

5.4.10 Determine whether each of the following words of length 15 is a codeword in C 15, where g(x) = 1 + x 4 + x 6 + x 7 + x 8.

(a) 011001011000010

(b) 000111010000110

(c) 011100000010001

(d) 111111111111111

5.5 Decoding 2 error-correcting BCH code

We describe a decoding scheme for the 2 error-correcting BCH codes constructed in the last section. Throughout this section, we shall identify a binary word of length r with the corresponding power of β.
< previous page | page_122 | next page >


page_123

< previous page | page_123 | next page >

Page 123

A parity check matrix for the (2r – 1,2r – 2r – 1,5) 2 error-correcting BCH code with generator g(x) = m 1(x)m 3(x) is H as defined in Lemma 5.4.5.

Assume the word w is received, and w ↔ w(x). Then the syndrome of w is

wH = [w(β), w(β 3)] = [s 1, s 3]

where s 1 and s 3 each are words of length r.

If no errors occurred in transmission, then the syndrome is wH = 0, so s 1 = s 3 = 0. If just one error occurred in transmission, then, the error polynomial is e(x) = x i thus wH = eH = [e(β), e(β 3)] = [β i,β 3i] = [s 1, s 3]. Therefore [image].

If two errors occurred in transmission, say in positions i and j, i ≠ j, then e(x) = x i + x j and wH = eH = [e(β), e(β 3)] = [s 1, s 3]. Thus the syndrome wH is given by

wH = [s 1,s 3] = [β i + β j, β 3i + β 3j].

We consider the resulting system of equations

β i + β j = s 1
β 3i + β 3j = s 3

Now we have the factorization

(β i + β j)(β 2i + β i + j) + β 2j) = β 3i + β 3j,

and

.

Therefore

.

Thus

.

Now β i and β j are roots of the quadratic equation

x 2 + (β i + β j)x + β i + j = 0

and hence roots of

.

Therefore we can find the positions of the errors by finding the solutions of this equation. The polynomial on the left side of this equation is called the error locator polynomial.
< previous page | page_123 | next page >


page_124

< previous page | page_124 | next page >

Page 124

Example 5.5.1 Let w ↔ w(x) be a received word with syndromes s 1 = 0111 = w(β) and s 3 = 1010 = w(β 3), where w was encoded using C 15. From Table 5.1 we have that s 1 ↔ β 11 and s 3 ↔ β 8. Then

.

We form the polynomial x 2 + β 11 x + β 2 and find that it has roots β 4 and β 13. Therefore we can decide that the most likely errors occurred in positions 4 and 13 (that is, e(x) = x 4 + x 13), so the most likely error pattern is

000010000000010.

Exercises

5.5.2 Verify by substitution that β 4 and β 13 are indeed solutions of the quadratic equation x 2 + β 11 x + β 2 = 0. Also check that the sum of the 4th and 13th rows of H in Table 5.3 is [s 1, s 3].

5.5.3 Find the roots in GF(24) of the following polynomials, if possible (use Table 5.1).

(a) x 2 + β 4 x + β 13

(b) x 2 + β 7 x + β 2

(c) x 2 + β 2 x + β 5

(d) x 2 + β 6

(e) x 2 + β 2 x

(f) x 2 + x + β 8.

We have arrived at a scheme for incomplete maximum likelihood decoding for the 2 error-correcting BCH codes. Let w be a received word. Clearly once an error pattern is determined then the algorithm is terminated.

Algorithm 5.5.4 (decoding 2 error-correcting BCH codes) Suppose the generator polynomial is m 1(x)m 3(x).

1. Calculate the syndrome wH = [s 1,s 3] = [w(β), w(β 3)].

2. If s1 = s3 = 0, conclude that no errors occurred. Decode c = w as the codeword sent.

3. If s 1 = 0 and s 3 ≠ 0 then ask for retransmission.

4. If [image] then correct a single error at position i, where s 1 = β i.

5. Form the quadratic equation

6. If equation (*) has two distinct roots β i and β j, correct errors at positions i and j.
< previous page | page_124 | next page >


page_125

< previous page | page_125 | next page >

Page 125

7. If equation (*) does not have two distinct roots in GF(2r), conclude that at least three errors occurred in transmission, and ask for a retransmission.

All examples and exercises that follow use C 15 whose parity check matrix is listed in Table 5.3 and generator polynomial g(x) listed in example 5.4.1.

Example 5.5.5 Assume w is received and the syndrome is wH = 01111010 ↔ [β 11,β 8]. Now

In this case equation (*) is x 2 + β 11 x + β 2 = 0, as is shown in Example 5.5.1. This equation has roots β 4 and β 13. So we correct errors in positions i = 4 and j = 13; in other words, the most likely error pattern is u = 000010000000010, and e(x) = x 4 + x 13 is the presumed error polynomial.

Example 5.5.6 Assume the syndrome is wH = [w(β), w(β 3)] = [β 3,β 9]. Then [image]. Therefore it is most likely that a single error occurred at position i = 3. The most likely error pattern is u = 000100000000000, and e(x) = x 3 is the error polynomial.

Example 5.5.7 Assume w = 110111101011000 is received. The syndrome is

wH = 01110110 ↔ [β 11,β 5] = [s 1, s 3].

Now [image]. To form the quadratic equation (*), we first calculate

So in this case, (*) becomes

x 2 + β 11 x + β 0 = 0.

Trying the elements of GF(24) in turn as possible roots, we come to x = β 7 and find

Now β 7 β j = 1 = β 15, so β j = β 8 is the other root. Therefore we correct errors at positions i = 7 and j = 8; that is u = 000000011000000 is the most likely error pattern. We decode v = w + u = 110111110011000 as the word sent.
< previous page | page_125 | next page >


page_126

< previous page | page_126 | next page >

Page 126

Example 5.5.8 Assume a codeword in C 15 is sent, and errors occur in positions 2, 6, and 12. Then the syndrome wH is the sum of rows 2, 6, and 12 of H, where w is the word received. Thus

wH = 00100011 + 00110001 + 11110011
= 11100001 ↔ [β 10,β 3] = [s1, s3].

Now [image]. We calculate

and then form the quadratic equation

x 2 + β 10 x + β 4 = 0.

By trying each of the elements of GF(24), we see that this equation has no roots in GF(24). Therefore IMLD for C 15 concludes correctly, that at least three errors occurred, and we request a retransmission.

Exercises

5.5.9 Messages are encoded using C 15. Determine, if possible the locations of the errors if w is received and the syndrome wH is a given in each part.

(a) 0100 0101

(b) 1110 1000

(c) 1100 1101

(d) 0100 0000

(e) 0000 0100

(f) 1010 0100

(g) 0011 1101

(h) 0000 0000

5.5.10 The code is C 15. Decode, if possible, each of the following received words w.

(a) 11000 00000 00000

(b) 00001 00001 00001

(c) 01000 10101 00000

(d) 11001 11001 11000

(e) 11001 11001 00000

(f) 11100 00000 00001

(g) 10111 00000 00000

(h) 10101 00101 10001

(i) 01000 01000 0000

(j) 01010 10010 11000

(k) 11011 10111 01100

(l) 10111 00000 01000

(m) 11100 10110 00000

(n) 00011 10100 00110
< previous page | page_126 | next page >


page_127

< previous page | page_127 | next page >

Page 127

Chapter 6
Reed-Solomon Codes

6.1 Codes over GF(2r)

We now turn to one of the most practical codes known, namely the Reed-Solomon codes. They are currently being used by both NASA and the European Space Agency; the codes chosen for use in compact discs also come from this family.

In the previous few sections we have extensively studied the 2-error correcting binary BCH codes. In fact, the Reed-Solomon codes are also BCH codes, but the digits in each codeword are no longer binary digits. This may seem strange as we have just finished praising the practical uses of these codes, and transmissions are always across binary channels. As will be shown, these codes do have a binary representation, but that is not how we shall first see the codes.

Before continuing, we develop some notation. Let GF(2r)[x] denote the set of all polynomials with coefficients from GF(2r). This set contains K[x], K = GF(2) = {0,1}, the set of all polynomials with binary coefficients. As before we can identify codewords c ∈ C, C a linear code over GF(2r) of length n, with polynomials c(x) ∈ GF(2r[x] having degree c(x) < n.

Recall we defined cyclic codes of length n in terms of roots of the corresponding polynomials. For instance, the (binary) 2-error correcting BCH code of length n = 2r – 1, can be described by c(x) ∈ C k if and only if β 1,β 2,β 3,β 4 are all roots of c(x) where c(x) ∈ K[x], degree (c(x)) < n and β is a primitive element in the field GF(2r). In this case gk(x) = m 1(x)m 3(x) is the generator polynomial for this cyclic code and c(x) ∈ C k if and only if c(x) = a(x)gk(x).

We can generalize this to codes over GF(2r) by choosing c(x) ∈ GF(2r)[x], instead. Again c(x) ∈ C if and only if {β 1,β 2,β 3,β 4}are all roots of c(x). Now however polynomials x + β, x + β 2,x + β 3, and in x + β 4 are in GF(2r)[x] and thus c(x) ∈ C if and only if g(x) = (x + β)(x + β 2)(x + β 3(x + β 4 divides c(x).

The binary code C K defined above is a BCH code. The code C over GF(2r), just defined, contains C K as a subcode and is an example of a Reed-Solomon code. In general, the code C K is said to be a subfield subcode of C K because C K ⊆ C and all words in C K have all their digits in the subfield K of GF(2r); that

< previous page | page_127 | next page >


page_128

< previous page | page_128 | next page >

Page 128

is C K = C ∩ K n.

Both of these codes CK and C are cyclic since c(x) ∈ C implies c(x) = xc(x) mod (1 + xn) is in C. This follows from the division algorithm and the fact that β i is a root of 1 + xn and xc(x). In fact, it is not hard to show that if g(x) generates a linear cyclic code of length 2r – 1 over GF(2r), then the generator of the binary subfield subcode is the polynomial gk(x) with the set of roots being the smallest set R that satisfies:

(a) if α is a root of g(x) then α ∈ R, and

(b) if α ∈ R then α 2 ∈ R.

Putting these observations together gives us the following result:

Theorem 6.1.1 Let α 1,α 2,...,α t be distinct non-zero elements of GF(2r). Then g(x) = (α 1 + x)(α 2 + x)...(α t + x) generates a linear cyclic code of length 2r – 1 over GF(2r).

Example 6.1.2 Let F = GF(24) constructed using 1 + x + x 4 (see Table 5.1). g(x) = (β + x)(β 2 + x) = β 3 + β 5 x + x 2 generates a linear cyclic code over F of length 15. The codeword corresponding to g(x) is of course β 3 β 51000000000000.

Also gk(x) = 1 + x + x 4 ↔ 110010000000000 is in this code and in fact generates the cyclic binary subfield subcode. To see this, using the notation above, we find R: from (a) β,β 2 ∈ R, and from (b) (β 2)2 = β 4 ∈ R and (β 4)2 = β 8 ∈ R; so R = {β,β 2,β 4,β 8} and thus gk(x)=(β 4 + x)(β 8 + x)g(x).

We summarize some basic results about cyclic codes over GF(2r):

Theorem 6.1.3 Let C be a linear cyclic code of length n over GF(2r). Then every codeword c(x) can be written uniquely as m(x)g(x) for some m(x) in GF(2r[x] of degree less than n – deg(g(x)). Also, g(x) divides f(x) if and only if f(x) is a codeword, and g(x) divides 1 + x n.

Corollary 6.1.4 Let g(x) have degree n – k. If g(x) generates a linear cyclic code C over GF(2r) of length n = 2r – 1, and dimension k then

is a generating matrix for C, and the number of codewords in C is (2r)k.

Remark The fact that |C| = 2rk follows from Theorem 6.1.3, since all of the polynomials m(x) in GF(2r)[x] of degree less than k give different codewords m(x)g(x); but there are 2rk such polynomials m(x) since each of the k coefficients in m(x) can be any one of the 2r field elements.
< previous page | page_128 | next page >


page_129

< previous page | page_129 | next page >

Page 129

Example 6.1.5 Construct GF(23) using 1 + x + x 3 with β as the primitive element. Let g(x) = (β + x)(β 2 + x) = β 3 + β 4 x + x 2. Then g(x) generates a linear cyclic code C over GF(23) of length 7. A generating matrix for C is

C has 85 codewords. The codeword corresponding to m(x) = 1 + β x + β 3 x 4 ↔ 1β00β 3 = m, for example, is m(x)g(x) ↔ mG = β 30β 4 ββ 61β 3.

Exercises

6.1.6 Construct GF(23) using 1 + x + x 3. Let g(x) = (1 + x)(β + x) generate a code C of length 7 over GF(23.

(a) How many codewords does C have?

(b) Construct a generating matrix G for C using Corollary 6.1.4.

(c) Encode the following messages using G:

(i) m(x) = 1 + β 6 x

(ii) m(x) = β 4 x 4

(iii) m(x) = 1 + x + x 2

(d) Find the generating polynomial of the cyclic binary subfield subcode.

6.1.7 Construct GF(24) using 1 + x + x 4. Let g(x) = (β + x)(β 2 + x)(β 3 + x)(β 4 + x) generate a linear cyclic code C over GF(24).

(a) How many codewords does C have?

(b) Construct a generating matrix G for C using Corollary 6.1.4.

(c) Encode the following messages using G:

(i) m(x) = 1 + β 7 x 10

(ii) m(x) = β 2 x + x 2

(iii) m(x) = 1 + x + x 2

(d) Find the generator polynomial gk(x) of the binary subfield subcode. Find m(x) such that gk(x) = m(x)g(x).

6.2 Reed-Solomon codes

In Section 6.1, generators for linear cyclic codes over GF(2r) were introduced, but no idea of the error correction capabilities of these codes was given. Here we shall address that question and then define the Reed-Solomon codes. We consider only Reed-Solomon codes, but most of the results about these codes apply directly to BCH codes, which are just subfield subcodes. We begin with a technical lemma.
< previous page | page_129 | next page >


page_13

< previous page | page_13 | next page >

Page 13

There are two quantities over which we have no control. One is the probability p that our BSC will transmit a digit correctly. The second is the number of possible messages that might be transmitted. The actual messages are not nearly as important as the number of possible messages. For example, only two messages were necessary before Paul Revere set off on his famous midnight ride.

Recall that for any set S, we denote by |S| the number of elements in S. Thus |Kn| = 2n from Exercise 1.2.2.

The two basic problems of coding then are:

1.9.1 Encoding We have to determine a code to use for sending our messages. We must make some choices. First we select a positive integer k, the length of each binary word corresponding to a message. Since each message must be assigned a different binary word of length k, k must be chosen so that |M| ≤ |Kk| = 2k. Next we decide how many digits we need to add to each word of length k to ensure that as many errors can be corrected or detected as we require; this is the choice of the codewords and the length of the code, n. To transmit a particular message, the transmitter finds the word of length k assigned to that message, then transmits the codeword of length n corresponding to that word of length k.

1.9.2 Decoding A word w in Kn is received. We now describe a procedure, called maximum likelihood decoding, or MLD, for deciding which word v in C was sent. There are actually two kinds of MLD.

1. Complete Maximum Likelihood Decoding, or CMLD. If there is one and only one word v in C closer to w than any other word in C, we decode w as v. That is, if d(v,w) < d(v1,w) for all v 1 in C, v 1 ≠ v, then decode w as v. If there are several words in C closest to w, i.e., at the same distance from w, then we select arbitrarily one of them and conclude that it was the codeword sent.

2. Incomplete Maximum Likelihood Decoding, or IMLD. Again, if there is a unique word v in C closest to w, then we decode w as v. But if there are several words in C at the same distance from w, then we request a retransmission. In some cases we might even ask for a retransmission if the received word w is too far away from any word in the code.

We will use IMLD for the examples and exercises in this section, and throughout most of the rest of the text. We emphasize that MLD does not always work; in particular, if too many errors were made in transmitting across the BSC, then MLD fails.

The codeword v in C closest to the received word w is the v for which the distance d(v,w) is least and hence, by Theorem 1.6.3, has the greatest associated probability φp(v,w) among all codewords, so is the most likely codeword sent. Example 1.6.4 demonstrates this. Since d(v,w) = wt(v+w), the weight of the

< previous page | page_13 | next page >


page_130

< previous page | page_130 | next page >

Page 130

Lemma 6.2.1 Let α 1,α 2,...,α t be non-zero elements of GF(2r). Then

Proof: If α i = α j for some i ≠ j then two rows of the matrix are identical, so the determinant is zero. Therefore for t ≥ i > j ≥ 1, (α i + α j) is a factor of the determinant, so ∏t≥i>j≥1(α i + α j) divides the determinant. Using the fact that both sides are both polynomials in α 1,...,α t of the same degree, we have shown that they differ by at most a common factor. This common factor must be 1 as can be seen by comparing the coefficients of [image] on both sides.

Example 6.2.2 Using Lemma 6.2.1 and GF(24) constructed using 1 + x + x 4 (see Table 5.1) we find that

Exercises

6.2.3 Find the following determinants using Lemma 6.2.1. Assume β is the primitive elements in GF(24) constructed using 1 + x + x 4 (Table 5.1).

We are now ready to present the main theorem concerning general BCH codes. The result is presented in its more general form, but is sufficient when considering Reed-Solomon codes.

Theorem 6.2.4 Let g(x) = (β m + 1 + x)(β m + 2 + x)...(β m+δ–1 + x) be the generator of a linear cyclic code C over GF(2r) of length n = 2r – 1, where β is a primitive element in GF(2r) and m is some integer. Then d(C) ≥ δ.
< previous page | page_130 | next page >


page_131

< previous page | page_131 | next page >

Page 131

Proof: For 1 ≤ i ≤ δ – 1, β m + i is a root of g(x), and thus the columns of

span C⊥. No linear combination of δ – 1 rows of this matrix is zero, as can be seen by evaluating the determinant of any δ – 1 rows, say

which is not zero since β is of order n = 2r – 1 and 1 ≤ j 1 < j 2... < j δ–1 ≤ n – 1. Therefore no linear combination of δ – 1 or fewer rows of the matrix is zero and so by Theorem 2.9.1 d(C) ≥ δ. Note, the columns of the H are linearly independent and thus H is a parity check matrix for C.

Remark The proof of this theorem applies to any cyclic binary linear code of length 2r – 1 with a generator containing β m + 1,...,β m + δ–1 among its roots. These binary codes are called primitive BCH codes and δ is called the designed distance of the code. Since they are binary subfield subcodes, CK ⊂ C, of Reed Solomon codes C we must have d(CK) ≥ δ for these codes as well.

A binary Reed-Solomon code RS(2r,δ) is a cyclic linear code over GF(2r) with generator g(x) = (β m + 1 + x)(β m + 2 + x)...(β m + δ–1 + x) for some integer m and some primitive element β of GF(2r).

So for example, the code constructed in Example 6.1.5 is an RS(8,3), and the code constructed in Exercise 6.1.7 is an RS(16,5).

Theorem 6.2.5 If C is an RS(2r,δ) then

(a) n = 2r – 1,
< previous page | page_131 | next page >


page_132

< previous page | page_132 | next page >

Page 132

(b) k = 2r – δ,

(c) d = δ, and

(d) |C| = 2rk.

Proof: (a) follows from Theorem 6.1.1, and (b) and (d) follow from Corollary 6.1.4. (Notice that a linear code over GF(2r) of dimension k has 2rk codewords, which is consistent with the result that a binary linear code (that is, a linear code over GF(2)) of dimension k has 2k codewords.) The fact that d ≥ δ follows from Theorem 6.2.4 and that d ≤ δ follows from the Singleton bound (see Theorem 3.1.7).

Remark Notice that since d = n – k + 1, Reed-Solomon codes are MDS (maximum distance separable) codes (see Theorem 3.1.8).

Before we do another example, notice that any RS(2r, δ) code C, can be represented as a binary code simply by replacing each digit in each codeword by the binary word of length r given by an index table of GF(2r). This code has length r(2r – 1) whereas the binary subfield subcode has length 2r – 1.

Let ĉ denote the binary representation of c ∈ C formed in this way and Ĉ denote the binary code formed from C, by this method. One of the reasons that Ĉ is so useful is that it performs well as a burst error correcting code (see Theorem 7.1.15).

Example 6.2.6 Let C be the RS(4,2) with g(x) = β + x and where GF(22) is constructed using 1 + x + x2. From Theorem 6.2.5, C has n = 3, k = 2, d = 2, and |C| = 16. From Corollary 6.1.4, a generating matrix for C is

From GF(22) we have that 0, 1, β, β 2 correspond to the vectors 00, 10, 01 and 11 respectively. The 16 messages u with their binary representation û along with the corresponding codewords c = uG of C and their binary representations ĉ are:

û | u | c = uG | ĉ | û | u | c = ug | û
0000 | 00 | 000 | 000000 | 0001 | 0β | 0β 2 β | 001101
1000 | 10 | β10 | 011000 | 1001 | 1β | βββ | 010101
0100 | β0 | β 2 β0 | 110100 | 0101 | ββ | β 21β | 111001
1100 | β 20 | 1β 20 | 101100 | 1101 | β 2 β | 10β | 100001
0010 | 01 | 0β1 | 000110 | 0011 | 0β 2 | 01β 2 | 001011
1010 | 11 | ββ 21 | 011110 | 1011 | 1β 2 | β0β 2 | 010011
0110 | β1 | β 201 | 110010 | 0111 | ββ 2 | β 2 β 2 β 2 | 111111
1110 | β 21 | 111 | 101010 | 1111 | β 2 β 2 | 1ββ 2 | 100111
< previous page | page_132 | next page >


page_133

< previous page | page_133 | next page >

Page 133

Exercises

6.2.7 Let C be the RS(4,3) with generator g(x) = (1 + x)(β + x).

(a) Find n,k,d and |C| for this code.

(b) Construct a generating matrix G for C using Corollary 6.1.4.

(c) Find all the codewords in C, their corresponding binary codewords in Ĉ and the corresponding messages (of course, encode the messages using G from (b)).

6.2.8 Let C be the RS(8,5) with generator g(x) = (1 + x)(β + x)(β 2 + x)(β 3 + x) using GF(23) constructed with 1 + x + x3.

(a) Find n,k,d, and |C| for this code.

(b) Find a generating matrix G for C using Corollary 6.1.4.

(c) Encode the following message using G to a codeword in C and then to a codeword in Ĉ:

(i) 10β 2

(ii) 111

(iii) β 2 β 4 β 6

6.2.9 Using the fields constructed in Exercise 5.1.15, find generator polynomials for the RS(2r, δ) code with the following values of r, δ and m:

(a) r = 2, δ = 3, m = 2

(b) r = 3, δ = 3, m = 2

(c) r = 3, δ = 5, m = 0

(d) r = 4, δ = 5, m = 0

(e) r = 5, δ = 7, m = 0

6.2.10 For each of the codes in Exercise 6.2.9, find the values of n,k,d and |C|.

From Theorem 6.2.5 we have that if C is an RS(2r, δ) then n = 2r – 1. Often one needs to have codes of lengths other than 2r – 1, but such codes can easily be formed from an RS(2r, δ) code. For any integer s with 1 ≤ s < 2r – δ, and for any RS(2r, δ) code C, form the shortened RS(2r, δ) code C(s) form C by taking all the codewords in C that have 0's in the last s positions and then deleting the last s positions.

Example 6.2.11 Let C be the RS(4,2) code of Example 6.2.6. The shortened RS(4,2) code C(1) (so s = 1) is formed by taking all codewords that have 0's in the last s = 1 position, namely

000, β10, β 2 β0 and 1β 20,

then deleting the last s positions. So

C(1) = {00,β1,β 2 β,1β 2}.

Alternatively, using the polynomial representation for an RS(2r, δ) code C, the shortened code C(s) is formed by the set of polynomials in C of degree less than

< previous page | page_133 | next page >


page_134

< previous page | page_134 | next page >

Page 134

n – s = 2r – 1 – s. So, if g(x) is the generator polynomial of C, then C(s) is the set of polynomials c(x) = a(x)g(x), where deg(a(x)) < k – s = 2r – δ – s (since deg(g(x)) = δ). Therefore, a generating matrix G(s) for the C(s) is given by

Comparing this to the generating matrix G of C given by Corollary 6.1.4, G(s) is the first k – s rows of G with the last s columns deleted.

So if C is an RS(2r,δ) code with parameters n,k and d then clearly we have that C(s) has length n(s) = n – s = 2r – 1 – s and dimension k(s) = k – s = 2r – δ – s.

To find the distance d(s) of C(s), notice that if c 1 and c 2 are codewords in C(s) then the distance between c 1 and c 2 is the same as the distance between the corresponding codewords c 100...0 and c 200...0 in C. Therefore d(C(s)) ≥ d(C) = δ. Also, from the Singleton bound in Theorem 3.1.7,

So we have that d(s) = δ, and from Theorem 3.1.8 we have that C(s) is also an MDS code. Therefore we have the following result.

Theorem 6.2.12 Let C be an RS(2r,δ) code and let C(s) be the shortened RS(2r,δ) code with parameters n(s), k(s), and d(s). Then

n(s) = 2r – 1 – s,

k(s) = 2r – δ – s,

d(s) = δ,

and C(s) is an MDS code (see Theorem 3.1.8).

Remark Other shortened RS(2r,δ) codes can be formed by deleting any set of s coordinates, instead of the last s coordinates as was presented here. Because RS(2r,δ) codes are MDS codes, any shortened RS(2r,δ) code will also have the properties described in Theorem 6.2.12.

Example 6.2.13 In Example 6.1.5 we constructed an RS(23,3) code C with generator polynomial g(x) = β 3 + β 4 x + x 2. The shortened RS(23,3) code C(2) has

< previous page | page_134 | next page >


page_135

< previous page | page_135 | next page >

Page 135

generating matrix

and has parameters n(2) = 5, k(2) = 3 and d(2) = 3. Notice that G(2) is formed by deleting the last s = 2 rows columns of the generating matrix G in Example 6.1.5.

6.3 Decoding Reed-Solomon codes

Since digits in RS(2r, δ) codes are elements from GF(2r), correcting a received word involves not only finding the locations of errors, but also the ''magnitudes" of those errors since the digits of a most likely error pattern come from GF(2r). With this in mind, we make the following definitions. The error locations of a received word are the coordinates in which the (most likely) error pattern is non-zero. The error locations are referred to by an error location number: if the jth coordinate of the received word is an error location then its error location number is β j (as with 2 error-correcting BCH codes, the coordinates are labeled 0, 1,...,n – 1). For example, steps 4 and 6 of Algorithm 5.5.4 find the error location numbers of the most likely error pattern when using the 2 error-correcting BCH code. The error magnitude of an error location i is the element of GF(2r) that occurs in coordinate i of the (most likely) error pattern. Since the 2 error-correcting BCH code defined in Chapter 5 is a code over GF(2), all error magnitudes must be 1 (the only non-zero element of GF(2)) and so are completely determined by the error locations. This is not the case for codes over GF(2r), r ≥ 2, so to decode the Reed-Solomon codes we need to find the error locations and their corresponding error magnitudes.

Example 6.3.1 Using RS(8,3) constructed in Example 6.1.5, if c = β 3 β 4 β 0 0000 is transmitted and w = β 3 β 4 β 5 0000 is received then the most likely error pattern is c + w = e = 00β 40000. So the error location number is β 2 and the corresponding error magnitude is β 4.

We shall now develop an algorithm for decoding the RS(2r,δ) code (and the corresponding BCH subfield subcode) with generator g(x) = (β m + 1 + x)(β m + 2 + x)....(β m + δ–1 + x) where β is a primitive element of GF(2r). Let t = [image] (δ – 1)/2 [image] as usual, and let a 1, ..., ae and b 1, ..., be be the error location numbers and their corresponding error magnitudes respectively, where e ≤ t. (So in Example 6.3.1, t = 1 and since one error occurred in the second position, a 1 = β 2 and b 1 = β 4). If e < t then it will be convenient to define a i = 0 for e + 1 ≤ i ≤ t, even though no such error locations exist. Then we can calculate δ – 1 syndromes s m + 1,...,s m + δ–1 which are defined by:

sj = w(β j) for m + 1 ≤ j ≤ m + δ–1.
< previous page | page_135 | next page >


page_136

< previous page | page_136 | next page >

Page 136

(Notice that this is the same definition of s 1 and s 3 used for the 2 error-correcting BCH code.) For m + 1 ≤ j ≤ m + δ – 1, β j is a root of g(x) and therefore is a root of all codewords, so

So the decoding problem is to find an effective way of solving 2e of the δ – 1 equations given by (6.1) for the 2e unknowns a 1,...,ae and b1,...,be. (Notice that 2e ≤ 2t ≤ δ – 1.) The difficulty in doing this lies in the non-linearity of the equations resulting from ai being raised to the jth power. However we shall now show how to easily find a polynomial whose roots are a 1,...,a e, just as we did in step 6 of Algorithm 5.5.4 when decoding the 2 error-correcting binary BCH code.

Let A = {a 1,...,ae} and define the error locator polynomial σA(x) to be the polynomial whose roots are precisely a 1,...,ae. So

Now define σ j to be coefficient of x j in σ A(x). Then after expanding the above product of σ A(x) we get

For any i with 1 ≤ i ≤ e, we can multiply both sides of (6.3) by [image], then substitute x = ai and sum both sides over i from 1 to t; then from (6.2) we have that σA(ai) = 0, and so we get

which may be rewritten as

But fortunately we know the values of s m + 1,s m + 2,...,s m + 2e, so we can substitute j = m + 1,...,m + e in turn to obtain e linear equations in the unknowns σ0,...,σe–1. These equations can most neatly be written in matrix form (where the ith row represents (6.5) with j = m + i) as follows:
< previous page | page_136 | next page >


page_137

< previous page | page_137 | next page >

Page 137

It is important to know that this linear system always has a non-trivial solution. Let the e × e matrix in (6.6) be called M. Then indeed M does have full rank. This can be seen by writing

Each of the 3 matrices has full rank by Lemma 6.2.1, since a 1,...,ae are distinct and a 1,...,ae, b 1,...,be are all non-zero. Therefore (6.6) can always be solved for σ0,...,σe–1. Notice also that if the decoder begins by assuming that e = t (of course the value of e is, at first, unknown to the decoder) then M is a t × (t + 1) matrix but will have rank e. This follows also by splitting M into the 3 matrices above and using the fact that we defined ai = 0 for e + 1 ≤ i ≤ t. Therefore the decoder now knows the value of e.

Now we can find a 1,...,ae by substituting the field elements into σA(x) = σ0 + α 1 x + ... + xe (which is now known), since the roots of σA(x) are precisely a 1,...,ae.

Now that a 1,...,ae are known, equations (6.1) form a linear system in the variables b 1,...,be which can now be solved. Again these equations can most easily be represented in matrix form as follows:

(Again, by Lemma 6.2.1 and since a 1,...,ae are distinct and non-zero, this matrix has full rank, so the linear system can always be solved for b 1,...,be.)

Therefore we have the following decoding algorithm for Reed-Solomon codes. In this algorithm, we define M′ be the extended matrix formed from M by adding a column e + 1 to M which is simply the right hand side of (6.6); that is,

Algorithm 6.3.2 (decoding RS(2r, δ)) Suppose that a codeword in an RS(2r, δ) code C with generator g(x) = (β m + 1 + x)...(β m + δ–1 + x) is transmitted and w is received. Let t = [image] (δ – 1)/2 [image]. Find the closest codeword in C to w as follows:
< previous page | page_137 | next page >


page_138

< previous page | page_138 | next page >

Page 138

1. Calculate s j = w(β j) for m + 1 ≤ j ≤ m + 2t.

2. Setting e = t, find the rank of the extended matrix M′.

3. Now let e be the rank of M′ and solve the linear system (6.6) for σ0,..., σe–1.

4. Find the roots of σA(x) = σ0 + σ1 x + ... + xe; these roots are the error location numbers a 1,...,ae.

5. Solve the linear system (6.7) for b 1,...,be; these are the error magnitudes corresponding to a 1,...,ae, so the most likely error pattern is completely determined.

Notice that no further row reduction of a matrix is required in Step 3 of Algorithm 6.3.2 since the matrix here is a submatrix of the one that is put into echelon form in Step 2. The following example makes this clear.

Example 6.3.3 Let

g(x) = (1 + x)(β + x)(β 2 + x)(β 3 + x) = β 6 + β 5 x + β 5 x 2 + β 2 x 3 + x 4

be the generator of an RS(23,5) code (so m = –1 and t = 2), where GF(23) is constructed using 1 + x + x 3. Suppose that the received word is

w = β 6 ββ 5 β 210β 2.

We shall now decode w using Algorithm 6.3.2.

1. Since m = –1 and δ = 5, we calculate the 4 syndromes s 0, s 1, s 2, and s 3 (in other words, calculate s i if β i is a root of g(x)).

2. Setting e = t = 2, the extended matrix M′ is

Row reducing M′ yields the matrix

which has rank 2.
< previous page | page_138 | next page >


page_139

< previous page | page_139 | next page >

Page 139

3. Since M′ has full rank, e = 2 and so we now solve the linear system

However, as observed above, we have already row reduced M in Step 2, so we have to solve

Then β 4σ1 + β 2 = 0 so σ1 + β 5, and σ0 + β 3 β 5 + β 3 = 0 so σ0 = 1.

4. We now know the error locator polynomial σA(x) = σ0 + σ1 x + x 2 = 1 + β 5 x + x 2. On substituting field elements into σA(x), we find that σA(β) = 0 and σA(β 6) = 0. Therefore

σA(x) = 1 + β 5 x + x 2=(β + x)(β 6 + x).

So the error location numbers are a 1 = β and a 2 = β 6.

5. Now we solve the following linear system:

or

Then β 5 b 2 = 1 so b 2 = β 2, and b 1 + b 2 = 1 so b 1 = β 6. Therefore the most likely error pattern is

e = 0β 60000β 2

and the most likely codeword is

c = w + e = β 6 β 5 β 5 β 2100.

Example 6.3.4 Let

g(x) = (1 + x)(β + x)(β 2 + x)(β 3 + x)(β 4 + x)(β 5 + x)
= 1 + β 4 x + β 2 x 2 + βx 3 β 12 x 4 + β 9 x 5 + x 6

be the generator of an RS(24, 7) code (so m = –1 and t = 3), where GF(24) is constructed using 1 + x + x 4 (see Table 5.1). Suppose that the received word is

w(x) = 1 + β 4 x + βx 3 + β 9 x 5 + x 6.
< previous page | page_139 | next page >


page_14

< previous page | page_14 | next page >

Page 14

error pattern u = v+w, Theorem 1.6.3 may be restated as follows:

that is, the most likely codeword sent is the one with the error pattern of smallest weight.

Thus the strategy in IMLD is to examine the error patterns v+w for all codewords v, and pick the v which yields the error pattern of smallest weight.

Example 1.9.3 Suppose |M| = 2, and we select n = 3 and C = {000, 111}. If v = 000 is transmitted, when will IMLD conclude this correctly, and when will IMLD incorrectly conclude that 111 was sent? We construct Table 1.1 as follows.

Received | Error Pattern |  | Decode
w | 000 + w | 111 + w | v
000 | 000* | 111 | 000
100 | 100* | 011 | 000
010 | 010* | 101 | 000
001 | 001* | 110 | 000
110 | 110 | 001* | 111
101 | 101 | 010* | 111
011 | 011 | 100* | 111
111 | 111 | 000* | 111
Table 1.1: IMLD table for Example 1.9.3.

The first column of Table 1.1 lists all possible words which might be received. This is all of K 3. The second and third columns list the error patterns v+w for each word v in the code C. Since IMLD will select the error pattern of smallest weight, we have put an asterisk beside the entry in column two or three of least weight. In the last column we record the word v in the code C corresponding to the column in which the asterisk was placed. This is the word v which IMLD will conclude was sent for each possible word received. Thus IMLD will conclude correctly that 000 was sent if 000, 100, 010, or 001 is received (first four rows of Table 1.1). And IMLD will conclude incorrectly that 111 was sent if 110, 101, 011, or 111 was received (last four rows of Table 1.1).

Example 1.9.4 Suppose |M| = 3, and we select C = {0000, 1010, 0111} with n = 4. We construct the IMLD Table 1.2, just as in Example 1.9.3 above, except that if two or more entries in the error pattern columns have the same smallest weight, then we do not place an asterisk in that row and record nothing (indicated by −) in the decoding column for v. This means, for IMLD, that we request a retransmission whenever there is a tie for smallest error pattern weight.
< previous page | page_14 | next page >


page_140

< previous page | page_140 | next page >

Page 140

1.

2.

So M has rank 2 and therefore the most likely error pattern has weight e = 2.

3. With e = 2, the linear system (6.7) becomes

but in Step 2 we row reduced this matrix:

Then β 12σ1 + β 7 = 0, so σ1 = β 10, and β 7σ0 + σ1 + β 9 = 0 so σ0 = β 6.

4. σA(x) = β 6 + β 10 x + x 2 = (β 2 + x)(β 4 + x). Therefore a 1 = β 2 and a 2 = β 4.

5.

so

Therefore b 2 = β 12 and b 1 = β 2. So the most likely error pattern is e = 00β 20β 120...0 and the most likely codeword sent is

c = w + e = 1β 4 β 2 ββ 12 β 9100...0.

Note that this decoding scheme is independent of the cyclic nature of the code, and thus will work for shortened RS(2r, δ) codes of length n as well.
< previous page | page_140 | next page >


page_141

< previous page | page_141 | next page >

Page 141

Exercises

6.3.5 Let C be the RS(24, 7) code with generator g(x) = (1 + x)(β + x)(β 2 + x)(β 3 + x)(β 4 + x)(β 5 + x) where GF(24) is constructed using 1 + x + x 4 (see Table 5.1); Decode the following received words which were encoded using C.

(a) 0β 3 ββ 5 β 3 β 2 β 6 β 10 β000000

(b) 1β 4 β 2 β0010ββ 5 β 3 β 20β10β

(c) β0β 70β 12 β 3 β 310000000

6.3.6 Let C be the RS(24,5) code with generator g(x) = (β + x)(β 2 + x)(β 3 + x)(β 4 + x) where GF(24) is constructed using 1 + x + x4 (see Table 5.1; notice here that m = 0). Decode the following received words that were encoded using C.

(a) 001β 800β 500000000

(b) 0β 100β 6 β 130β 8 β 11 β 3 β 500000

(c) β 40100β 2 β 5 β 12 β 14000000

6.3.7 Take the RS(24, 5) code in exercise 6.3.6 and form the shortened code C(4) of length n = 11 (and dimension k = 7). Decode the following received words, that were encoded using C.

(a) 001β 800β 50000

(b) 0β 100β 6 β 130β 8 β 11 β 3 β 50

(c) β 40100β 2 β 5 β 12 β 1400

6.3.8 Let C be the RS(24, 9) with generator g(x) = (1 + x)(β + x)...(β 7 + x) and with GF(24) constructed using 1 + x + x 4 (see Table 5.1). Find the most likely error pattern for received words that were encoded using C and have the following syndromes.

(a) s 0 = β 2,s 1 = β 3,s 2 = β 4,s 3 = β 5,s 4 = β 6,s 5 = β 7,s 6 = β 8 and s 7 = β 9.

(b) s 0 = β 9,s 1 = β 13,s 2 = β 7,s 3 = β 4,s 4 = β 12,s 5 = β 4,s 6 = β 8 and s 7 = β 2.

(c) s 0 = 1,s 1 = 1,s 2 = 1,s 3 = 1,s 4 = 1,s 5 = 1,s 6 = 1 and s 7 = 1.

(d) s 0 = β 10,s 1 = β 3,s 2 = β 13,s 3 = β 3,s 4 = β 12,s 5 = β 5,s 6 = β 13 and s 7 = β 3.

(e) s 0 = β 12,s 1 = β 8,s 2 = 0, s3 = β 7,s 4 = β 13,s 5 = β 4,s 6 = β 13 and s 7 = 1.

(f) s 0 = β 2,s 1 = 0, s2 = 0, s3 = β 2,s 4 = 0, s5 = 0, s6 = β 2 and s 7 = 0.

6.4 Transform approach to Reed-Solomon codes

There is an alternate approach to the construction and decoding of Reed-Solomon codes which is sometimes referred to as the transform approach. It is based on an alternate representation of vectors in K n. Rather than think of a vector as representing the coefficients of a polynomial, we consider a vector as representing a function from a set S to a field F = GF(2r).
< previous page | page_141 | next page >


page_142

< previous page | page_142 | next page >

Page 142

We first develop this approach and show that it gives a different generator matrix for Reed-Solomon codes.

Example 6.4.1 Let S = GF(23), constructed using 1 + x + x 3 with primitive element β. First consider f : S → {0, 1} where f(0) = 0, f(1) = 0, f(β) = f(β 2) = f(β 4) = 1, f(β 6) = f(β 3) = f(β 5) = 0. Then f(x) can be prescribed by the vector vf = (f(0), f(1), f(β), ..., f(β 6)) = (0, 0, 1, 1, 0, 1, 0, 0).

Example 6.4.2 Let S = GF(23). Consider a function g : S → S, defined by

vg = (g(0), g(1), g(β), ..., g(β 6))
= (β 4, 0, 1, β 2, 1, β, 0, 0).

In this case, we can also represent g(x) as a polynomial,

g(x) = β 4 + β 2 x + β 3 x 2 + x 3

Two polynomials p(x) and q(x) represent the same function from S to GF(2r), S ⊆ GF(2r), if and only if p(α) = q(α) for all α ∈ S. If we consider all polynomials of degree ≤ k – 1 with coefficients from GF(2r), or equivalently the vector form of these polynomials as functions from S ⊆ GF(2r) to GF(2r), we see that they form a vector space and the basis is the set of polynomials {1, x,x 2,...,x k–1}. We will refer to such a vector space as a function space on S.

Theorem 6.4.3 The set of all functions from S to F = GF(2r) represented by polynomials of degree ≤ k – 1 from a function space of dimension k with basis {1,x,x 2,...,x k–1}.

Proof: Certainly every polynomial of degree ≤ k – 1 is in the span of {1,x,x 2,...,x k–1}. All we need to prove is that each function has a unique representation. Suppose that p(x) and q(x) are equal as functions on S. Then p(α) = q(α) for all α ∈ S. But then p(α) – q(α) = 0 and p(x) – q(x) is a polynomial of degree < k with n roots n ≥ k, which is impossible unless p(x) – q(x) = 0 and thus p(x) and q(x) are equal as polynomials.

Example 6.4.4 Let F = GF(23), constructed using 1 + x + x 3 and consider all polynomials of degree ≤ 2. A basis for this function space is {1,x,x 2} with the corresponding vector forms.

Clearly any polynomial p(x) = a 0 + a 1 x + a 2 x 2 considered as a function can be represented in vector form by a matrix product:
< previous page | page_142 | next page >


page_143

< previous page | page_143 | next page >

Page 143

Recall that a Maximum Distance Separable code (MDS-code) is a linear code (n,k,d) with the distance d = n – k + 1.

Theorem 6.4.5 The function space on S ⊆ GF(2r) of all polynomials of degree ≤ k – 1 with coefficients from GF(2r) form a linear (n,k,n – k + 1) MDS code, where n = |S| ≤ 2r.

Proof: We choose a subset S ⊆ GF(2r), |S| = n and consider the function space of all polynomials p : S → GF(2r) of degree ≤ k – 1. Clearly the length of each vector form (and hence of the code) is n and the dimension from Theorem 6.4.3 is k, k ≤ n. To establish the minimum distance, note that any polynomial p(x) of degree ≤ k – 1 has at most k – 1 different roots; hence the vector form of p(x) has at most k – 1 zeros and thus has weight at least n – k + 1. By the Singleton bound (Theorem 3.1.7), d ≤ n – k + 1 for any linear code, so therefore d = n – k + 1.

The subset S = {α ∈ F | α n = 1}, is said to be the set of nth roots of unity in F = GF(2r). Then n is necessarily a divisor of 2r – 1 (but n need not equal 2r – 1), and so n is odd. S consists of all roots of 1 + xn in F. An element β ∈ S is said to be a primitive nth root of unity (in GF(2r)) if S = {1,β,β 2,...,β n–1}. This generalizes the idea of a primitive element of a field and will allow for the construction of cyclic Reed-Solomon codes of lengths n that divide, but are not necessarily equal to 2r – 1. In fact everything done previously in this chapter for codes of length 2r – 1 = n where β is a primitive element remains true when β is just a primitive nth root of unity.

Example 6.4.6 Let F = GF(24) constructed using 1 + x + x 4, with primitive element β. Then the 5th roots of unity are {1,β 3,β 6,β 9,β 12}, and the set of 3rd roots of unity are {1,β 5,β 10}. In this case β 3 is a primitive 5th root of unity, and β 5 is a primitive 3rd root of unity.

We wish to construct cyclic Reed-Solomon codes, but first we need to establish that two polynomials will represent the same function on S if and only if they are equivalent (mod 1 + xn).

Theorem 6.4.7 Let p(x), q(x) ∈ GF(2r)[x], and S ⊆ GF(2r) be the set of nth roots of unity. Then p(x) and q(x) represent the same function f : S → GF(2r) (i.e., p(β i) = q(β i), for all β i ∈ S) if and only if p(x) ≡ q(x) (mod 1 + xn).

Proof: Let q(x) = h(x)(1 + x n) + p(x), where degree (p(x)) < n. Then q(β i) = h(β i)(β in + 1) + p(β i) = p(β i) because β i is a root of (1 + x n). Conversely if q(β i) = p(β i), β i is a root of p(x) – q(x). Hence
< previous page | page_143 | next page >


page_144

< previous page | page_144 | next page >

Page 144

Theorem 6.4.8 Let S be the set of nth roots of unity in GF(2r). The function space of all polynomials in GF(2r[x] of degree ≤ k – 1 on S forms a cyclic (n,k,n – k + 1) code over GF(2r).

Proof: In order for the code C to be cyclic, if v p = (p(1), p(β), ..., p(β n-1)) ∈ C, we must have that (p(β), p(β 2), ..., p(β n–1), p(1)) is also in C. But, note that p(β x) is still a polynomial of degree ≤ k – 1 and so p′(x) = p(β x) ∈ C. But (p′(1), p′(β), ..., p′(β n–1)) = (p(β), p(β 2), ..., p(β n–1), p(1)).

Example 6.4.9 Consider GF(23) constructed using 1 + x + x 3. Let p(x) = β 4 + β 2 x + β 3 x 2 + x 3 with vector form (0,1,β 2,1,β,0,0). Then the shift of this vector (1,β 2,1,β,0,0,0) corresponds to the function p(β x) = β 4 + β 3 x + β 5 x 2 + β 3 x 3 = (β 4 + x)(β 5 + x)(β 6 + x)β 3.

If we have a polynomial V(x) = V 0 + V 1 x + ... + V n–1 x n–1 we say that v(x) = v 0 + v 1 + x + ... + v n–1 x n–1 is the transform of V(x) if V(β j) = ∑Vi β ji = vj for j = 0, 1, ..., n – 1. In terms of matrix notation this is equivalent to (V 0, V 1, ..., V n–1)A = (v 0, v 1, ... v n–1) where A = (aij) and aij = β ij; β a primitive nth root of unity in GF(2r). The matrix A is usually referred to as the finite Fourier transform (or finite field transform). It has an inverse A–1. Thus we have

(V 0, V 1, ..., V n) = (v 0, v 1, ..., vn)A–1

or

We saw earlier in Lemma 6.2.1 that A was invertible but we produce an alternate proof of this, by showing that A–1 transforms v back to V.

Theorem 6.4.10 Let β be a primitive nth root of unity. If v j = V(β j) for V(x) = V 0 + V 1 x + ... + V n–1 x n–1 then V i = v(β–i), where v(x) = v 0 + v 1 x + ... + v n–1 x n–1.

Proof: because

Note that (1 + xn) = (1 + x)(1 + x + ... + x n–2 + x n–1) so if β k–i ≠ 1 then it is a root of 1 + x + ... + x n–2 + x n–1. Also, recall that n divides 2r – 1, so is odd.

< previous page | page_144 | next page >


page_145

< previous page | page_145 | next page >

Page 145

As we see, given a vector of values (v 0, v 1, ..., v n–1) we can recover the coefficients of the polynomial V(x) = V 0 + V 1 x + ... + V n–1 x n–1. This is essentially what the decoding algorithm presented in Section 6.3 is trying to do.

Theorem 6.4.11 Let S be the set of nth root of unity in GF(2r). The function space of all polynomials of degree < n – δ + 1 on S is a cyclic MDS code with generator polynomial g(x) = (β + x)(β 2 + x)...(β δ–1 + x), where β is a primitive nth root of unity.

Proof: The polynomial function C(x) whose vector form corresponds to c(x) = [image]. Since c(β n–i) = 0 for i = n –δ + 1, n – δ + 2, ..., n – 1 the coefficient of xi is zero in C(x) and thus C(x) has degree < n – δ + 1.

In summary, we have produced an alternate method of constructing a RS(2r, δ) code when n = 2r – 1, one which results in a different generator matrix (and a different view of the information digits).

Example 6.4.12 Let β be a primitive element in GF(23) constructed using 1 + x + x 3. Consider a RS(23, 5) code (Exercise 6.2.8) with generator polynomial g(x) = (1 + x)(β + x)(β 2 + x)(β 3 + x) = β 6 + β 5 x + β 5 x 2 + β 2 x 3 + x 4 which corresponds to (β 6,β 5,β 5,β 2,1,0,0}. The transform of g(x) is the polynomial

Since (g(β 0), g(β 1), ..., g(β 6)) = (0, 0, 0, 0, 1, β, β 4) then G(x) = g(β 7–1)x + g(β 7–2)x 2 + g(β 7–3)x 3 = β 4 x + βx 2 + x 3 = x(β 4 + βx + x 2).

It is easy to check that G(x) represents a function with vector form

(G(β 0), G(β 1), ..., G(β 6)) = (β 6,β 5,β 5,β 2,1,0,0).

In this case we think of this RS(23, 5) code as the function space of all polynomials with degrees of all terms between 1 and 3. Clearly the basis for these polynomials is {x,x 2,x 3} and the corresponding generator matrix for this function space is:

Thus G(x) = β 4 x + βx 2 + x 3 if and only if its vector form is

Now we consider how this approach can help with the decoding of Reed-Solomon codes. Recall that if g(x) is the generator polynomial for an RS code and w(x) is the received word then w(x) = c(x) + e(x), where c(x) = a(x)g(x)

< previous page | page_145 | next page >


page_146

< previous page | page_146 | next page >

Page 146

and e(x) is the error polynomial. Let W(x), C(x), and E(x) be the transforms of w(x), c(x), and e(x), respectively. The transform is a linear mapping. So

Since g(x) has δ – 1 consecutive roots β k, k = m + 1, m + 2, ..., m + δ – 1, we have c(β k) = 0, and thus the syndromes s n–k are w(β n–k) = e(β n–k) = E k for these values of k. That is, the syndromes give us δ – 1 of the coefficients of the transform of e(x). What we want to do is find the remaining coefficients. For this we need the error locator polynomial!

σ(x) was defined so that σ(β k) = 0 if and only if ek ≠ 0 (recall that σ(β k) =0 if and only if β k is an error location number, which happens if and only if e(x) is non-zero in the kth position). Since E(β k) = ek, we have that σ(β k)E(β k) = 0 for all k and thus

σ(x)E(x)≡0 (mod 1 + x n)

and

(since σ(x) is a polynomial of degree at most t = [image] (δ – 1)/2 [image]), thus equating coefficients of the xt + k we have

0 = σt Ek + σ t–1 E k + 1 + σt–2 E k + 2 + ... + σ0 Ek + t.

Since we know δ – 1 consecutive values Ek (that is, the syndromes sn–k), we can compute the coefficients σi and use this to generate all values of Ek.

Example 6.4.13 Let σ(x) = σ0 + σ1 x + x 2 and E(x) = E 0 + E 1 x + ... + E 6 x 6. Then σ(x)E(x) ≡ 0 (mod 1 + x 7) if and only if

Ek = σ1 E k + 1 + σ0 E k + 2, k = 0, 1, ..., 6

Example 6.4.14 Consider Example 6.3.3. Suppose w = (β 6,β,β 5,β 2,1,0,β 2). Since d = 5, t ≤ 2 and E 0 = w(β 0) = 1, E 6 = w(β) = β 3, E 5 = w(β 2) = β 3, E 4 = w(β 3) = 1 and σ(x) = x 2 + σ1 x + σ0. We solve for σ1, σ0 as in Example 6.3.3, and find σ1 = β 5, σ0 = 1. Thus, E k = β 5 E k + 1 + E k + 2.

Since (E 0, E 6, E 5, E 4) = (1,β 3,β 3,1), we see that
< previous page | page_146 | next page >


page_147

< previous page | page_147 | next page >

Page 147

Now we know that the transform of e(x) is E(x) = Σ Ek xk where

(E 0, E 1,...,E 6) = (1, β 2, 0, β 2, 1, β 3, β 3).

At this point our decoding algorithm will depend on the encoding procedure. Since E(x) = 1 + β 2 x + β 2 x 3 + x 4 + β 3 x 5 + β 3 x 6, we know that the most likely error vector is e = (E(β 0), E(β 1),..., E(β 6)) = (0, β 6,0,0,0,0,β 2) and thus the most likely codeword is c = w + e = (β 6,β 5,β 5,β 2,1,0,0).

On the other hand if we used the generator matrix of Example 6.4.12 we would not need to find the error vector, but simply compute all values of w(β k),k = 0,...,6 to obtain the most likely message directly. That is, find the transform of w(x) and add it to the transform of e(x).

and thus

So C(x) = β 4 x + βx 2 + x 3. Hence our information digits are (β 4,β,1). We leave it as an exercise to show that c = (β 6,β 5,β 5,β 2,1,0,0) is the vector form of C(x).

Exercises

6.4.15 Show that C(x) = β 4 x + βx 2 + x 3 has vector form (β 6,β 5,β 5,β 2,1,0,0), β the primitive element in GF(23) constructed using 1 + x + x 3.

6.4.16 Given GF(23) constructed using 1 + x + x 3, find the generator matrix for the MDS code of length 7 for the function space of all polynomials defined on S = GF(23)\{0} with basis

(a) {x,x 2,x 3}

(b) {1,x,x 2,x 3,x 4}

(c) {x,x 3,x 6}

6.4.17 Show that all codes in the previous exercise are cyclic. Find the generator polynomial for each code.

6.4.18 Given GF(23) constructed using 1 + x + x 3, for each G(x) find the corresponding vector form vg in the function space.

(a) G(x) = x + β x 3

(b) G(x) = 1 + x2 + x 4
< previous page | page_147 | next page >


page_148

< previous page | page_148 | next page >

Page 148

6.4.19 Given GF(23) constructing using 1 + x + x 3 with primitive element β, find the coefficients of the polynomial G(x) given the values vg.

(a) vg = (β 3,β,β 4,0,β 6,β 5,β 2)

(b) vg = (β 4,β 2,β,β 3,0,β 6,1)

6.4.20 For Exercises 6.3.5, 6.3.6, and 6.3.8, use the transform approach to compute the most likely error pattern and decode.

6.5 Berlekamp-Massey algorithm

The following algorithm for finding the error locator polynomial is faster than solving the linear system (6.6). The algorithm is essentially the algorithm of Berlekamp and Massey for calculating the error locator polynomial σ(x) given the syndromes s j = w(β j) for m + 1≤ j ≤m + 2t.

Let σ R(x) = 1 + σ t–1 x + σ t–2 x 2+···+ σ 0 x t; that is, σ R(x) is the ''reverse" of the error locator polynomial σ(x). Let s(x) = 1 + s m+1 x + s m+ 2 x 2 +···+ s m+ 2t x 2t be the syndrome polynomial. Using the division algorithm, we can write

σR(x)s(x) = q(x)x 2t + 1 + r(x)

with deg(r(x))) ≤ 2t. But, since the coefficients of x t + 1,...,x 2t in σR(x)s(x) are all zero from (6.4), in fact deg(r(x)) ≤ t.

The version of the Berlekamp-Massey algorithm below produces a polynomial P 2t(x) satisfying P 2t(x)s(x) = q 2t(x)x 2t + 1 + r 2t(x) with deg(P 2t(x)) ≤ t, deg(r 2t(x)) ≤ t and P 2t(0) = 1 (if such exists). This is clearly enough to show that P 2t(x) = σR(x).

(The algorithm actually produces a sequence of polynomials Pi(x) and integers Di such that if P i(x)s(x) = q i(x)x i + 1 + r i(x) with deg(ri(x)) ≤ i, then deg (P i(x)) ≤ i – [image] D i/2 [image] and deg (r i(x)) ≤ i – [image] (1 + D i)/2 [image]. Moreover Pi(x) is a combination of P i–1(x) and some specific previous polynomial P zi–1(x).)

Let q i(x) = q i,0 + q i,1x+···+ q i,2t–1–i, x 2t–1–i and let p i(x) = x 2t + 1–i P i(x) = p i,0 + p i,1 x+···+ p i,e x e. At step i, the algorithm calculates q i(x), p i(x), the integers D i, and the subscripts z i (which are used to determine which other polynomial P zi(x) besides P i(x) is needed for the next calculation). The proof that this version of the algorithm works is straightforward, so is left for the exercises. We now give the algorithm precisely.

Algorithm 6.5.1 (Berlekamp-Massey; to find the error locator polynomial) Let w be a received word that was encoded using a generator g(x) having as roots the consecutive powers β m + 1,...,β m + 2t of β. Decode w as follows.

1. Calculate s j :=w(β j) for m + 1≤j≤m + 2t.
< previous page | page_148 | next page >


page_149

< previous page | page_149 | next page >

Page 149

2. Define

Let D–1 := –1 and D 0 := 0; and let z 0 := –1.

3. For 1≤i≤2t, recursively define qi(x), Pi(x), Di, and zi as follows.

(a) If q i–1,0 = 0, then let

(b) If qi,0 ≠ 0, then let

which can be truncated to have degree at most 2t – 1 – i; and let

If e ≤ t errors have occurred during transmission then p 2t(x) = σ R(x) has degree e; the error locator polynomial is σ(x) = p 2t,e + p 2t,e–1 x +···+ p 2t,1 x e–1 + x e, which has e distinct roots.

Although we did not actually calculate the remainder polynomials ri(x) in this algorithm, they certainly are part of the theory involved. In fact r 2t(x) := p 2t(x)q–1(x) mod x 2t + 1 has a further use in decoding. Either r(x) := r 2t(x) or p(x) := r 2t(x) – p 2t(x) is called the error evaluator polynomial because, together with [image] it can be used to calculate the error magnitudes bj, given the error locations aj.
< previous page | page_149 | next page >


page_15

< previous page | page_15 | next page >

Page 15

Received | Error Pattern |  |  | Decode
w | 0000 + w | 1010 + w | 0111 + w | v
0000 | 0000* | 1010 | 0111 | 0000
1000 | 1000 | 0010 | 1111 | —
0100 | 0100* | 1110 | 0011 | 0000
0010 | 0010 | 1000 | 0101 | —
0001 | 0001* | 1011 | 0110 | 0000
1100 | 1100 | 0110 | 1011 | —
1010 | 1010 | 0000* | 1101 | 1010
1001 | 1001 | 0011 | 1110 | —
0110 | 0110 | 1100 | 0001* | 0111
0101 | 0101 | 1111 | 0010* | 0111
0011 | 0011 | 1001 | 0100* | 0111
1110 | 1110 | 0100* | 1001 | 1010
1101 | 1101 | 0111 | 1010* | 0111
1011 | 1011 | 0001* | 1100 | 1010
0111 | 0111 | 1101 | 0000* | 0111
1111 | 1111 | 0101 | 1000* | 0111
Table 1.2: IMLD table for Example 1.9.4.

Exercises

1.9.5 Let |M| = 2, n = 3, and C = {001, 101}. If v = 001 is sent, when will IMLD conclude this correctly, and when will IMLD incorrectly conclude that 101 was sent?

1.9.6 Let |M| = 3 and n = 3. For each word w in K 3 that could be received, find the word v in the code C = {000, 001, 110} which IMLD will conclude was sent.

1.9.7 Construct the IMLD table for each of the following codes.

(a) C = {101, 111, 011}

(b) C = {000, 001, 010, 011}

(c) C = {0000, 0001, 1110}

(d) C = {0000, 1001, 0110, 1111}

(e) C = {00000, 11111}

(f) C = {00000, 11100, 00111, 11011}

(g) C = {00000, 11110, 01111, 10001}

(h) C = {000000, 101010, 010101, 111111}

Recall that we have to choose n and C (1.9.1). Some choices are better than others. We list three important criteria for measuring good choices:

1. Longer words take more time to transmit and decode, so n should not be too large; that is, the information rate should be as close to 1 as possible.

2. With many messages being received per second, if |C| is large, say a few thousand or so, the procedure for IMLD described in this section would be too time consuming to implement. Fortunately, certain clever choices of C admit much slicker and faster methods for IMLD.
< previous page | page_15 | next page >


page_150

< previous page | page_150 | next page >
Page 150 The following is a quick derivation for a formula for bj in terms of (p(x) or) r(x) and [image] Let Then Since [image] to get a polynomial in x with degree clearly less than the degree of σ(x). Then But [image] unless [image] Note also that so that Since [image] the numerator can be viewed as [image] instead of [image] to get Example 6.5.2 Consider Example 6.3.4 with the syndromes s 0 = β 7, s 1 = β 0, s 2 = β 9, s 3 = β 12, s 4 = β 9, s 5 = β 7. Working step by step through Algorithm 6.5.1, we obtain the following. We begin by setting
< previous page | page_150 | next page >


page_151

< previous page | page_151 | next page >

Page 151

Let i = 1. Since q0,0 = β 7 ≠ 0, we use step 3(b):

(q0(x) + β 7 q–1(x))/x = β 3 + x + β 13 x 2 + β 14 x 3 + β 14 x 4 + β 14 x 5

which is truncated to degree 2t – i – 1 = 4 to give

and since D 0 ≥ D–1,zi = i – 1 = 0.

Before proceeding, we shall adopt a more concise format by representing the polynomials by their corresponding words. Then the information we have found so far is represented by the following table.

Proceeding from i = 2 to i = 2t = 6 we obtain the following table.

So finally we obtain σ(x) by reading p2t(x) = p6(x) backwards:

σ(x) = β 6 + β 10 x + x 2.

Example 6.5.3 Let C be the RS(24,9) code with generator g(x) = (1 + x)(β + x)...(β 7 + x) and with GF(24,9) constructed using 1 + x + x 4 (Table 5.1). Suppose that w is the received word and the syndromes of w are:

s 0 = β 12,s 1 = β 9,s 2 = β 6,s 3 = β 3,s 4 = β 5,s 5 = β 12,s 6 = β 6,s 7 = β 6.

Using Algorithm 6.5.1 and the notation described in Example 6.5.2, we obtain the error locator polynomial as follows.
< previous page | page_151 | next page >


page_152

< previous page | page_152 | next page >

Page 152

Therefore the error locator polynomial is

σ(x) = β 13 + β 10 x + β 13 x 2 + β 12 x 3 + x 4.

Remark Notice that in Algorithm 6.5.1, at every step Zi is either i–1 or Zi–1. Therefore at each step we need only store q i–1, p i–1, D i–1, Z i–1, q zi–1, and p zi-1, and not everything calculated so far, as is suggested by the tables of Examples 6.5.2 and 6.5.3. Clearly this is an important practical consideration, but to describe the algorithm it is convenient to display all the calculations in one table.

Exercises

6.5.4 Let C be the RS(24,9) with generator g(x) = (1 + x)(β + x)...(β 7 + x) and with GF(24) constructed using 1 + x + x 4 (Table 5.1). Use Algorithm 6.5.1 to find the error locator polynomial for received words that were encoded using C and have the following syndromes. (See also Exercise 6.3.8 which has the same syndromes.)

(a) s 0 = β 2,s 1 = β 3,s 2 = β 4,s 3 = β 5,s 4 = β 6, s5 = β 7,s 6 = β 8, and s 7 = β 9.

(b) s 0 = β 9,s 1 = β 13,s 2 = β 7,s 3 = β 4,s 4 = β 12, s5 = β 4,s 6 = β 8, and s 7 = β 2.

(c) s 0 = 1,s 1 = 1,s 2 = 1,s 3 = 1,s 4 = 1,s 5 = 1,s 6 = 1, and s 7 = 1.

(d) s 0 = β 10,s 1 = β 3,s 2 = β 13,s 3 = β 3,s 4 = β 12,s 5 = β 5,s 6 = β 13, and s 7 = β 3.

(e) s 0 = β 12,s 1 = β 8,s 2 = 0,s 3 = β 7,s 4 = β 13,s 5 = β 4,s 6 = β 13, and s 7 = 1.

(f) s 0 = β 2,s 1 = 0,s 2 = 0,s 3 = β 2,s 4 =0,s 5 =0,s 6 =β 2, and s 7 = 0.

6.5.5 (proof of Berlekamp-Massey)

(a) Prove recursively that deg (P i(x)) ≤i– [image] D i/2 [image] (nothing that P i(0) = 1).

(b) Prove recursively that deg (R i(x)) ≤i– ⌉(1 + D i)/2 [image], after first showing that the choice of q i(0) forces the coefficient of x i in P i(x)q-1(x) to be zero.

(c) Prove that since all the Dj are distinct, that at least one of the Dj, j≤i, must be at least i; and hence that either Di≥i or Dzi≥i.
< previous page | page_152 | next page >


page_153

< previous page | page_153 | next page >

Page 153

(d) When D 2t ≥ 2t, show that deg(P 2t(x))≤t, and that since deg(R 2t(x)) ≤ t note that at least t consecutive coefficients of P 2t(x)q-1(x) are zero (meaning that at least t consecutive Newton's identities are satisfied).

6.6 Erasures

An erasure is an error for which the error location number is known but the error magnitude is not known. An erasure location number is the location number of an erasure. Knowledge of the error location number may come from the physical reading of the signal being received (the received digit not looking like a zero or a one), but can also come from the structure of the code. For example, suppose that C is an RS(2r,δ) code and Ĉ is the binary representation of C. Define Ĉ′ to be the binary code formed from C by adding a parity check digit to the binary representation of each digit in each codeword of C.

Example 6.6.1 Let C be the RS(4,2) defined in Example 6.2.6. To form Ĉ′ the digits 0,1,β and β 2 of codewords in C are replaced by 000,101,011 and 110 respectively (the third digit in each of these words is a parity check digit). So the codeword in Ĉ′ that corresponds to the codeword β 1 0 in C is 011101000.

Exercises

6.6.2 Let C be the RS(4, 3) with generator g(x) = (1 + x)(β + x) (see Exercise 6.2.7). Find all of the codewords in Ĉ′.

Since each digit in a codeword c from an RS(2r,δ) code C is represented by a binary word of length r + 1 in the corresponding codeword ĉ in Ĉ′ has length (2r 1)(r + 1). Also, since each non-zero digit in c is replaced by a word of even weight in ĉ′ ĉ′ consists of 2r – 1 words of length r + 1, each of which should have even weight. Therefore if one of these 2r – 1 groups has odd weight in a received word [image] then we know an error has occurred among these r + 1 digits. We could try decoding [image] by decoding w, the word having digits in GF(2r) that corresponds [image] to the closest codeword in C. Knowing that errors occurred in a group of r + 1 digits corresponds to knowing one error location number of w, which therefore is an erasure.

Example 6.6.3 Using the code Ĉ′ defined in Example 6.2.6, suppose that 011 100 000 is received. Then we know errors occurred among the second group of 3 digits (since this group of 3 digits has odd weight), so β 1 is an erasure location number. Since this position of w is an erasure, we may as well replace it with 0 (to make it easier to find the syndromes), so we now try to decode w = β 0 0 to the closest codeword in C, knowing that one error location number is a1 = β.
< previous page | page_153 | next page >


page_154

< previous page | page_154 | next page >

Page 154

Theorem 6.6.4 Let C be an RS(2r,δ) which is used to transmit messages and let w be a received word containing ∈ erasures and e errors which are not erasures. Then w can be decoded correctly if

2e + ∈ ≤δ–1.

Proof: Let B be the set of erasure locations and let A be the set of error locations; so A–B is the set of error locations which are not erasure locations. Define

to be the erasure locator polynomial. Then we can express the error locator polynomial as

σA(x) = σB(x)σA–B(x).

Finding the unknown error locations requires finding the roots of σA–B(x). Providing we can remove the effect of the erasures on the syndromes, finding the roots of σA–B(x) can simply be done with Algorithm 6.3.2 (or Algorithm 6.5.1) using ''modified" syndromes.

To see what the modified syndromes should be, we slightly alter the development of Algorithm 6.3.2. Write

σB(x) = B 0 + B 1 x + ··· + B∈–1 x∈–1 + x∈

and

σA–B(x) = A 0 + A 1 x + ... + A e–1 x e–1 + xe.

In the same way as (6.4) was obtained, multiply both sides of σB(x)σA–B(x) = σA(x) by [image] (where m + 1 ≤ j ≤ m + δ–1, and a 1,...,a e + ∈ are the error location numbers) and substitute x = ai then sum both sides from i = 1 to e + ∈ to obtain

So we form the modified syndromes by defining

Since we know the values of sj for m + 1 ≤ j ≤ m + δ–1 and since B0,...,B∈–1 are known, [image] as*j is known for m + 1 ≤ j ≤ δ–1–∈. But since 2e + ∈ ≤ δ–1,

< previous page | page_154 | next page >


page_155

< previous page | page_155 | next page >

Page 155

2e ≤ δ–1–∈ so as when writing (6.6), we can solve the linear system formed from (6.8):

for the e unknown values A 0 A 1,...,A e–1

We can now decode Reed-Solomon codes with erasures by modifying Algorithm 6.3.2 in the obvious manner suggested by the proof of Theorem 6.6.4.

Algorithm 6.6.5 (decoding erasures in RS(2r,δ)) Suppose that c is a codeword in an RS(2r,δ) code C with generator g(x)=(β m + 1 + x)···(β m + δ–1 + x) that is transmitted and w is received containing ∈ erasures with erasure location numbers being the elements of B={a 1,...a∈}. Let σB(x=(a 1 + x)···(a∈ + x)=B 0 + B 1 x +···+ B∈–1 x∈–1 + x∈ be the erasure locator polynomial. The error locator polynomial σA(x)=σA–B(x)σB(x) can be found by finding σA–B(x) = A 0 + A 1 x + ... + A e–1 + x e as follows:

1. Calculate sj=w(β j) for m + 1 ≤ j ≤ m + δ–1.

2. Calculate [image] = B 0 s j + B 1 s j+1 +···+ B∈–1 S j+∈–1 + S j+∈ for m + 1 ≤ j ≤ m + δ––1–∈.

3. Solve the linear system (6.9) for A 0,A 1,...,A e–1.

Then decoding w can be completed using steps 4 and 5 of Algorithm 6.3.2.

As would now be expected, the modified syndromes of Algorithm 6.6.5 can be used to adapt Algorithm 6.5.1 for finding the error locator polynomial when erasures are included.

Algorithm 6.6.6 (Berlekamp-Massey decoding with erasures) Suppose C is an RS(2r,δ code with generator g(x)=(β m + 1 + x)···(β m + δ–1 + x) and let w be a received word. Let σB(x)= B 0 + B 1 x + ... + x∈ be the erasure locator polynomial of w. Modify Algorithm 6.5.1 for finding the error locator polynomial of w as follows:

1. Calculate sj=w(β j) for m + 1 ≤ j ≤ m + δ – 1.

2. Calculate [image] = B 0 s j + B 1 s j + 1 + ··· +B∈–1 + s j + ∈ for m + 1 ≤ j ≤ m + δ – 1 – ∈.

3. Define

and define p–1(x), p 0(x), D–1, D 0, and z 0 as in step 2 of Algorithm 6.5.1.
< previous page | page_155 | next page >


page_156

< previous page | page_156 | next page >

Page 156

4. Repeat step 3 of Algorithm 6.5.1, except that i is now restricted to 1 ≤ i ≤ δ – 1 – ∈, to produce σA–B(x). Then the error locator polynomial is σ(x) = σB(x)σA–B(x).

Remark To complete the decoding, b 1,b 2,...,b∈ + e can be found using step 5 of Algorithm 6.3.2 using the original syndromes; of course (6.7) is now a system of ∈ + e linear equations.

In the following examples we will use the code Ĉ′ to transmit messages. Then from the structure of the code, some erasures can be recognized.

Example 6.6.7 Messages are encoded using Ĉ′ where C is the RS(24,6) code with generator g(x) = (1 + x)(β + x)···(β 4 + x), where GF(24) is constructed using 1 + x + x 4. Decode the received word

= 11101 11001 00101 00000 00110 10010 0...0.

The only erasure location number is β 1, and so

σB(x) = β + x.

We can now use Algorithm 6.6.6 to find the error locator polynomial for

w = β 100β 20β 6 β 140...0.

where we have set the digit in the position corresponding to the erasure equal to zero (this makes calculating the syndromes easier). Since w(x) = β 10 + β 2 x 2 + β 6 x 4 + β 14 x 5, we find that s 0 = β 5,s 1 = 0,s 2 = β 3,s 3 = β 4, and s 4 = β 3. Since B 0 = β and ∈ = 1, we find (from Step 2) that [image], [image], [image] and [image]. Proceeding as in Algorithm 6.5.1, but now using the modified syndromes, we obtain:

So σA–B(x) = β 8 + β 11 x + x 2. Hence
< previous page | page_156 | next page >


page_157

< previous page | page_157 | next page >

Page 157

Therefore the error location numbers are a 1 = β, a 2 = β 3, and a 3 = β 5.

Of course we can then complete the decoding by finding b 1, b 2, and b 3: we use step 5 of Algorithm 6.3.2, the original syndromes and (6.7):

or

which gives b 1 = β 12,b 2 = 1 and b 3 = β 3. So w(x) is decoded to

and [image] is decoded to

11101 11110 00101 10001 00110 10001 0...0.

Example 6.6.8 Messages are encoded using the code Ĉ′ as defined in Example 6.6.7. Decode

The erasure locator polynomial is

σB(x) = (β + x)(β 3 + x) = β 4 + β 9 x + x 2.

We decode

w = β 100β 20β 6 β 140...0

to a codeword in C (again, the digits in w occurring in positions corresponding to erasures have been set equal to 0). Since w(x) = β 10 + β 2 x 2 + β 6 x 4 + β 14 x 5, s 0 = β 5, s 1 = 0, s 2 = β 3, s 3 = β 4 and s 4 = β 3. Therefore, from Step 2 of Algorithm 6.6.6, [image], [image], and

< previous page | page_157 | next page >


page_158

< previous page | page_158 | next page >

Page 158

So σA–B(x) = β 5 + x, so σA(x) = (β + x)(β 3)(β 5 + x) and the error magnitudes can be calculated as in Example 6.6.7.

When introducing the code Ĉ′ with C an RS(2r,δ) code, we noted that Ĉ′ has minimum distance at least 2δ and so can correct all binary error patterns of weight at most δ – 1. Using Algorithm 6.6.6 will find the closest codeword to a received word if at most δ – 1 binary errors occur during transmission of ĉ′ as the following argument shows. Suppose that u is the most likely binary error pattern, that wt(u) ≤ δ – 1, and that u causes ∈ erasures and e error which are not erasures. Then since at least 2 errors must be made in [image] to cause an error which is not an erasure in w, we have that 2e + ∈ ≤ δ – 1. Then by Theorem 6.6.4, w (and so therefore [image]) will be decoded correctly.

However, if Algorithm 6.6.6 decodes [image] to a codeword ĉ that is further than δ – 1 from [image] then the reader should be critical of the answer as we have no guarantee that ĉ′ is the closest codeword to

Exercises

6.6.9 Let C be the RS(23,5) with generator g(x) = (1 + x)(β + x)(β 2 + x)(β 3 + x where GF(23) is constructed using 1 + x + x 3. Decode the following received words that were encoded with ĉ′ by using Algorithm 6.6.6:

(a) 1011 1010 1111 0011 1001 0000 0000

(b) 1011 0000 1000 0011 1010 0011 1001

(c) 0101 1000 0000 1100 1100 1100 0101

(d) 0000 1010 1011 1101 0111 1001 0000

6.6.10 Use the code Ĉ′ defined in Example 6.6.7 and use Algorithm 6.6.6 to decode the following received words:

(a) 11101 11110 11010 00111 11110 10100 10110 10100 0...0

(b) 11000 00000 01010 11111 11011 00000 10001 00101 0...0

(c) 00000 10000 10000 10000 00101 10100 11101 0...0

6.6.11 Let C be the RS(24, 7) with generator g(x) = (β + x)...(β 6 + x) where GF(24) is constructed using 1 + x + x 4. Decode the following received word which was encoded using Ĉ′:

01011 11011 10001 11011 01001 11101 11110 10000 0...0.
< previous page | page_158 | next page >


page_159

< previous page | page_159 | next page >

Page 159

Chapter 7
Burst Error-Correcting Codes

7.1 Introduction

Until now we have only been interested in designing codes that correct randomly distributed errors. However there exist channels in which errors are likely to occur very close to each other. For example, a possible source of noise in a compact disc is a scratch across the disc: all digits occurring at the scratch may be either altered or erased causing a group of errors to occur close together. Also, sunspots are a factor in disrupting messages sent from satellites to earth, so once such activity starts in the sun we can expect several errors to occur close together. We call many errors occurring together in this fashion a burst of errors.

Suppose the polynomial e(x) corresponding to the word e can be factored as e(x) = x k e′(x), where e′(0) = 1, then we say the burst length of e is deg(e′(x)) + 1. So the burst length is the number of digits from the first 1 in e to the last 1 in e.

A related concept is the cyclic burst length of a word e. The word e ∈ K n is said to have cyclic burst length ℓ, if the minimum degree of x k e(x) mod (1 + x n) for k = 0, 1,..., n – 1 is ℓ – 1.

Example 7.1.1 Let n = 7, and e = 0101100. Then e(x) = x + x 3 + x 4 = x(1 + x 2 + x 3) and e therefore has burst length 4. If we consider xk e(x) mod (1 + x 7), that is all cyclic shifts of e, we see that x 6 e(x) mod (1 + x 7) has the smallest degree, 3, and thus the cyclic burst length is also 4.

On the other hand e = 1000100 ↔ 1 + x 4 has burst length 5 but 1 + x 3 = x 3(1 + x 4) mod (1 + x 7) and thus e has cyclic burst length 4.

Up until now we have always assumed that the most likely error pattern is the error pattern that has least weight. This is based on the fundamental assumption that errors occur independently. In various actual situations this assumption is not valid and thus our error correction strategy must change.

Recall that in MLD for a linear code, we take as coset representatives the words of least weight in the cosets, and say that such a code is t error-correcting precisely when all the words of weight at most t are in different cosets of the code.

< previous page | page_159 | next page >


page_16

< previous page | page_16 | next page >

Page 16

3. If many errors are made in transmission, MLD will not work. That is, the word MLD will conclude was sent will not be the same as the actual word sent. So C should be chosen so that the probability that MLD will work is very high. (We will consider this probability in the next section.)

Thus we claim that the main goal of coding theory is to find sets C of words which are adequate when judged by the criteria above. Most of the rest of our efforts will be devoted to this goal.

1.10 Reliability of MLD

Suppose n and C have been chosen. We now give a procedure for determining the probability θp(C,v) that if v is sent over a BSC of probability p then IMLD correctly concludes that v was sent.

Find the set L(v) of all words in Kn which are closer to v than to any other word in C. Then θp(C,v) is the sum of all the probabilities φp(v,w) as w ranges over L(v). That is,

Note that L(v) is precisely the set of words in Kn for which, if received, IMLD will correctly conclude that v was sent. We can find L(v) from the IMLD table constructed as in the last section. In each row of the table where v is decoded in the last column, the word w in the first column of that row is in L(v). And these are all the words in L(v).

Also observe that θp(C,v) is the sum over the words w in L(v) of the probabilities of the error patterns v + w occuring during transmission.

θp can be used to compare two codes, judging them by the third criterion in the previous section. However, it should be noted that θp(C,v) as defined ignores the possibility of retransmission, when the received word is equidistant from two codewords. This does lead to some anomalies (such as θp(Kn,v) > θp(C,u), for any v in Kn and u in C, where C is the parity check code formed from Kn), but is a reasonable first approximation for a measure of reliability. Certainly θp(C,v) is a lower bound for the probability that v is decoded correctly.

Example 1.10.1 Suppose p =.90, |M| = 2, n = 3, and C = {000, 111}, as in Example 1.9.3. If the word v = 000 is sent, we compute the probability that IMLD will correctly conclude this after one transmission. From Table 1.1, v = 000 is decoded in the first four rows, so the set L(000) (words in K 3 closer to v = 000 than to 111) is

L(000) = {000, 100, 010, 001}.
< previous page | page_16 | next page >


page_160

< previous page | page_160 | next page >

Page 160

In the correction of burst error patterns, we take as coset representatives the error patterns with burst of least length in each coset. So a linear code is an ℓ burst error-correcting code if all the words of burst length at most ℓ are in different cosets of the code. In general, if C is t error-correcting and ℓ burst error-correcting, then t ≤ l (Why?) and this inequality may be strict (see Exercises 7.1.5, 7.1.6, and 7.1.7).

Similarly, a linear code is an ℓ-cyclic burst error correcting code if all error patterns of cyclic burst length at most ℓ are in different cosets.

Example 7.1.2 Consider all non-zero cyclic burst error patterns of length at most 3 in K 15. Each such non-zero error pattern is e(x) = xk e′(x), k = 0, 1,..., 14 for e′(x) ∈ {1,1 + x,1 + x 2,1 + x + x 2}. Thus there are 4.15 = 60 such error patterns.

Example 7.1.3 Let g(x) = 1 + x + x 2 + x 3 + x 6 be a generator for a cyclic linear code of length 15 and dimension 9. Clearly this code is not a 3 error-correcting code since there are 576 error patterns of weight 3 or less but there are only 64 cosets. However there are only 61 error patterns of cyclic burst length 3 or less (see Example 7.1.2) so this code may be, and in fact is, a 3 cyclic burst error-correcting code (see Exercise 7.1.4). This can be checked by calculating the syndromes of xk e′(x) mod g(x), k = 0, 1,..., 14, and e′(x) ∈ {1,1 + x,1 + x 2,1 + x + x 2}.

Exercises

7.1.4 Verify that the cyclic burst error patterns of length 3 in K 15 occur in different cosets for the code in Example 7.1.3.

7.1.5 Show that g(x) = 1 + x 2 + x 4 + x 5 generates a 2 cyclic burst error-correcting linear code C of length 15. Is C a 2 error-correcting code?

7.1.6 Show that g(x) = 1 + x 3 + x 4 + x 5 + x 6 generates a 3 cyclic burst error-correcting linear cyclic codes of length 15. Is C a 3 error-correcting code? (Hint: Use the Hamming bound.)

7.1.7 Show that g(x) = 1 + x 4 + x 6 + x 7 + x 8 generates a 2 error-correcting, 4 cyclic burst error-correcting linear cyclic code of length 15.

As we have noticed, if C is a t error-correcting, ℓ burst error-correcting code then ℓ ≥ t. The following result gives an upper bound for the value l. A better upper bound can be obtained (see Exercise 7.1.10) but this result will be sufficient for our purpose.

Theorem 7.1.8 If C is an ℓ burst error-correcting linear code of length n and dimension k then ℓ ≤ n – k.

Proof: Let C be an ℓ burst error-correcting linear (n,k) code. Then no two error patterns, each with burst of length at most ℓ occur together in the same coset. Therefore no two words in which all the 1's occur in the first ℓ positions can occur in the same coset. As there are 2ℓ such words, there must be at least 2ℓ cosets, and so n – k ≥ ℓ.

< previous page | page_160 | next page >


page_161

< previous page | page_161 | next page >

Page 161

Exercises

7.1.9 Verify that each of the codes in Exercises 7.1.5, 7.1.6 and 7.1.7 satisfy the bound of Theorem 7.1.8.

7.1.10 Show that if C is an ℓ burst error-correcting linear code of length n and dimension k then ℓ ≤ (n – k)/2. (Hint: show that any error pattern with burst of length 2ℓ can be written as the sum of two error patterns with bursts e 1 and e 2 respectively, each burst of which has length at most ℓ. Then show that e 1 + e 2 is not a codeword.)

It turns out that there is an excellent decoding algorithm for correcting cyclic burst error patterns if we use a linear cyclic code. Suppose that C is an ℓ burst error-correcting linear cyclic code with generator polynomial g(x) of degree n – k. Recall from the discussion before Example 4.3.7 that

is a parity check matrix for C, where, for 1 ≤ i ≤ n – 1, ri is the word of length n – k that corresponds to the polynomial ri(x) = xi mod g(x). By using H, if w(x) = c(x) + e(x) is the received word, then it was shown that w ↔ w(x) has syndrome wH = s ↔ s(x) = w(x) mod g(x). There are two important facts that make C and H useful for decoding cyclic burst error patterns.

1. If e ↔ e(x) is an error pattern with burst of length at most ℓ, then ℓ ≤ n – k by Theorem 7.1.8. Therefore for some i with 0 ≤ i ≤ n – 1, the ith cyclic shift e i of e has all its 1s in the first ℓ positions (that is, xi e(x) mod 1 + xn ↔ ei has degree less than ℓ).

2. It is easy to calculate the syndrome of wi, the ith cyclic shift of w, since

So to correct w = c + e we can proceed as follows. For each i with 0 ≤ i ≤ n – 1 in turn we calculate the syndrome si ↔ xi s(x) mod g(x) (by fact 2) of wi until we find one, say sj, such that deg(sj(x)) < ℓ. So to correct w = c + e we

< previous page | page_161 | next page >


page_162

< previous page | page_162 | next page >

Page 162

can proceed as follows. For each i with 0 ≤ i ≤ n – 1 in turn we calculate the syndrome si ↔ xi s(x) mod g(x) (by fact 2) of wi until we find one, say sj, such that deg(sj(x)) < ℓ. At this point, since the first n – k rows of H form an identity matrix, we know that ej = sj00...0, formed by appending k 0s to the end of sj, has syndrome ej H = sj. Since C is cyclic, we therefore know that wj = cj + ej, where cj is the codeword formed by the jth cyclic shift of c. Thus, we can shift ej back through j cyclic shifts to find e = e 0, the most likely cyclic burst error pattern. This gives the following algorithm.

Algorithm 7.1.11 (decoding cyclic burst error patterns) Suppose w is a received word that was encoded using an ℓ-cyclic burst error correcting cyclic linear code with generator polynomial g(x).

1. Calculate the syndrome polynomial s(x) = w(x) mod g(x).

2. For each i ≥ 0, calculate si(x) = xi s(x) mod g(x) until a syndrome polynomial sj(x) is found with deg(sj(x)) ≤ ℓ – 1. Then the most likely cyclic burst error pattern is e(x) = xn–j sj(x) mod (1 + xn).

Example 7.1.12 g(x) = 1 + x + x 2 + x 3 + x 6 generates a 3-cyclic burst error-correcting linear cyclic code of length 15. Use Algorithm 7.1.11 to decode the received word 111100100001010, assuming that cyclic burst error patterns are most likely to occur.

1. s(x) = 1 + x + x 2 + x 3 + x 6 + x 11 + x 13 mod g(x)
= 1 + x 3 + x 4 + x 5

2. s1(x) = xs(x) mod g(x) = 1 + x 2 + x 3 + x 4 + x 5,
s2(x) = x 2 s(x) mod g(x) = 1 + x 2 + x 4 + x 5,
s3(x) = x3 s(x) mod g(x) = 1 + x 2 + x 5,
s4(x) = x4 s(x) mod g(x) = 1 + x 2,

and deg(s 4(x)) = 2 ≤ l – 1. Therefore the most likely error pattern is

So the most likely codeword sent is

Exercises

7.1.13 g(x) = 1 + x + x 2 + x 3 + x 6 generates a 3 cyclic burst error correcting linear cyclic code C of length 15. Decode the following received words that were encoded using C.
< previous page | page_162 | next page >


page_163

< previous page | page_163 | next page >

Page 163

(a) 101101110001000

(b) 001101100010101

(c) 100110101010011

(d) 101101000010111

(e) 000000111110000.

7.1.14 g(x) = 1 + x 2 + x 4 + x 5 generates a 2 cyclic burst error-correcting linear cyclic code C of length 15. Decode the following received words that were encoded using C.

(a) 010101000010010

(b) 011010010010100

(c) 001101000000100

(d) 000100010100101

(e) 000000011111001.

Reed-Solomon codes also have good burst error correction capability. Recall that if C is an RS(2r,δ) code then Ĉ is the binary representation of C (see Example 6.2.6).

Theorem 7.1.15 Let C be an RS(2r,2t + 1) code. Then Ĉ is an ℓ burst error correcting code, where ℓ ≥ r(t – 1) + 1.

Proof: Any burst error pattern e of length at most r(t – 1) + 1 produces a word [image] = ĉ + e where d(w, c) ≤ t. So w is decoded to the codeword c in the RS(2r, 2t + 1) code and therefore ĉ is the closest codeword in Ĉ to [image].

Two places where Reed-Solomon codes are used are in compact discs, where scratches on the disc cause bursts of errors, and in space communications by NASA and the ESA, where sunspots cause bursts of errors in the transmissions which are in the form of electromagnetic waves. Under such circumstances, assuming that errors occur in bursts is a better model than the assumption of random errors.

Example 7.1.16 The RS(8, 5) code of Exercise 6.2.8 will correct all burst error patterns of length at most r(t – 1) + 1 = 4.

7.2 Interleaving

One method for improving the burst error correcting capability of a code is to make use of interleaving. This technique rearranges the order in which code digits are transmitted. Until now, messages m 1, m 2,... have been encoded to corresponding codewords c 1, c 2,... and these codewords are transmitted in turn. Suppose now that the first s codewords are selected, then the first digit from each of these s codewords is transmitted, followed by the second digits, third digits, and so on. Once all ns digits in the first s codewords have been transmitted in this order, the same process is applied to the second set of s codewords to be sent, then the third set, and so on. This rearrangement of the order in which codeword digits are transmitted is called interleaving to depth s. More formally, interleaving

< previous page | page_163 | next page >


page_164

< previous page | page_164 | next page >

Page 164

c 1, c 2,... to depth s requires that for i = 0,1,2... in turn the codeword digits are transmitted in the order:

It is probably simpler to see this ordering by listing each of the codewords c is + 1, ..., c is + s row by row (see Table 7.1), then transmitting these digits column by column.

Table 7.1: Interleaving to depth s.

Example 7.2.1 Let C be the linear code with generating matrix

With no interleaving, the codewords

c 1 = 100110, | c 4 = 010101,
c 2 = 010101, | c 5 = 100110, and
c 3 = 111000, | c 6 = 111000

would be sent one after the other, so the code digits would be sent in the following order:

100110 010101 111000 010101 100110 111000.

If these codewords are interleaved to depth 3, then the first digits of c 1, c 2 and c 3, namely 1, 0 and 1 are transmitted first, followed by their second digits 0, 1 and 1, and so on. So the digits in c 1, c 2 and c 3 are transmitted in the following order:

101 011 001 110 100 010.

Notice that in this string, the digits in c 1 appear in positions 3i + 1, for 0 ≤ i ≤ 5. Notice also that with c 1, c 2 and c 3 written in rows as in Table 7.1, the digits are simply transmitted column by column. After these 21 digits have been transmitted, a similar rearrangement of the digits in c 4, c 5 and c 6 is used and so those digits are transmitted in the order:

011 101 001 110 010 100.
< previous page | page_164 | next page >


page_165

< previous page | page_165 | next page >

Page 165

What is the effect of interleaving to depth s on the burst error correcting capabilities of a code C? Notice that if the first digit of a codeword c is the ith digit transmitted, then the remaining digits occur in positions i + s, i + 2s,...,i (n – 1)s. Suppose that C is an ℓ burst error correcting code. If C is interleaved to depth s then any burst of errors of length at most sℓ during transmission will produce a burst error pattern of length at most ℓ in c, so c will be decoded correctly, providing that this is the only burst error pattern that affects c. Therefore we have the following result.

Theorem 7.2.2 Let C be an ℓ burst error correcting code. If C is interleaved to depth s then all bursts of length at mosts ℓ will be corrected, providing that each codeword is affected by at most one burst of errors.

Remark The provision that each codeword is affected by at most one burst of errors essentially requires that bursts of errors are separated by periods of error free transmission which are sufficiently long to avoid two bursts of errors affecting one block of s codewords. So choosing s to be large increases the burst length that Theorem 7.2.2 guarantees can be corrected, but also increases the length of error free transmission surrounding the burst required by Theorem 7.2.2.

Example 7.2.3 The code C in Example 7.2.1 is a 1 error-correcting code. When interleaved to depth 3 it corrects all bursts of length 3.

Exercises

7.2.4 Encode the messages m 1 = 1000, m 2 = 0110, m 3 = 1110, m 4 = 0011, m 5 = 0110, m 6 = 0001, then find the string of digits transmitted if the code is interleaved to depth s, where

(a) s = 1

(b) s = 2

(c) s = 3

7.2.5 In Example 7.1.3, it was stated that g(x) = 1 + x + x 2 + x 3 + x 6 is the generator for a linear cyclic code C of length 15 that is a 3 cyclic burst error correcting code. Use the generating matrix

for C to encode the messages m 1(x) = 1, m 2(x) = x 2, m 3(x) = 1 + x, m 4(x) = 1 + x 2, m 5(x) = x 3 n, and m 6(x) = 1. Find the string of digits transmitted if C is interleaved to depth s, where
< previous page | page_165 | next page >


page_166

< previous page | page_166 | next page >

Page 166

(a) s = 1

(b) s = 2

(c) s = 3.

In each case, what does Theorem 7.2.2 say about the burst error correcting capability of the code?

In practice, interleaving to depth s has the disadvantage that s codewords must be encoded before any of them are transmitted. This drawback can be overcome by using s-frame delayed interleaving, which lists the digits in each codeword as in Table 7.2 (compare this with Table 7.1) and again transmits the digits column by column. The array in Table 7.2 has n rows. Each codeword ci has exactly one digit ci,j in row j (for 1 ≤ i ≤ n) and ci,j + 1 is one row below and s columns across from ci,j (for 1 ≤ j ≤ n – 1).

Table 7.2: s-frame delayed interleaving.

Clearly there is some initialization process that must take place when using s-frame delayed interleaving, since for example if s ≥ 1 then the only entry defined in the first column of Table 7.2 is c 1,1. To ensure that each column of Table 7.2 contains n digits we place a 0 in any position that contains no codeword digits. The following example makes this clear, though the 0's introduced in this initialization process have been replaced by * to distinguish them from the 0 codeword digits.

Example 7.2.6 Consider again the six codewords c 1,...,c 6 of Example 7.2.1. If we use 1-frame delayed interleaving, the Table 7.2 becomes

where we have placed * in the positions where the initialization process defines the entry 0. The string of digits transmitted is

If we use 2-frame delayed interleaving, then Table 7.2 becomes
< previous page | page_166 | next page >


page_167

< previous page | page_167 | next page >

Page 167

and the string of digits transmitted is

It is easy to get the analogue of Theorem 7.2.2 for s-frame delayed interleaving.

Theorem 7.2.7 Let C be an ℓ burst error correcting code. If C is s-frame delay interleaved then all bursts of length ℓ(sn + 1) will be corrected, providing that each codeword is affected by at most one burst of errors.

Exercises

7.2.8 Use s-frame delayed interleaving and the codewords found in Exercise 7.2.4 to find the string of digits transmitted when

(a) s = 1

(b) s = 2.

7.2.9 What string of digits is transmitted if 0-frame delayed interleaving is used?

7.2.10 Prove Theorem 7.2.7.

In practice, the encoding of a message often uses 2 codes. For example, 2 codes are used in the encoding of music on to compact discs (see Section 7.3) where both codes are Reed-Solomon codes, and 2 codes are used by NASA and the European Space Agency, where one code is a Reed-Solomon code, the other a convolutional code (see Section 8.2). Interleaving to depth s is an important technique in this 2 step encoding, as we shall now see.

Let C 1 be an (n 1,k 1,d 1) linear code and C 2 an (n 2,k 2,d 2) linear code. Cross-interleaving of C 1 with C 2 is done as follows. Messages are first encoded using C 1 and the resulting codewords are interleaved to depth k 2. The columns formed (as in Table 7.1) in this interleaving process are all of length k 2, so can now be regarded as messages and encoded using C 2. The codewords resulting from this second encoding can themselves be interleaved to any depth s, or s-frame delay interleaved.

The main advantage of this 2 step encoding is the following. C 2 can be used to detect d 2 – 1 errors, rather than to correct errors. If errors are detected in a codeword in C 2 then all digits in this codeword are flagged and treated as digits that may be incorrect. The codewords in C 1 are then considered. Notice that if we know that n 1 – d 1 + 1 digits in a codewords c in C 1 are correct then we can always find the remaining d 1 – 1 digits. (This is so because it is impossible for

< previous page | page_167 | next page >


page_168

< previous page | page_168 | next page >

Page 168

another codeword in C 1 to agree with c in the n 1 – d 1 + 1 correct digits since all codewords disagree in at least d 1 positions.) Therefore, if each codeword in C 1 contains at most d 1 – 1 flagged digits, and if we assume that all incorrect digits are flagged, then the codewords will be decoded correctly.

Example 7.2.11 Let C 1 and C 2 be the codes with generating matrices

respectively. Then, (n 1,k 1,d 1) = (8,4,4) and (n 2,k 2,d 2) = (6,3,3). We shall encode the messages m 1 = 1000, m 2 = 1100 and m 3 = 1010 by the cross interleaving of C 1 with C 2, C 2 being interleaved to depth s = 3 = d 1 – 1. Encoding m1, m2 and m 3 with C 1 gives:

c 1 = m 1 G 1 = 10001110,

c 2 = m 2 G 1 = 11000011, and

c 3 = m 3 G 1 = 10100101.

The columns produced by interleaving these codewords to depth k 2 = 3 produces the messages

111, 010, 001, 000, 100, 101, 110, and 011.

These are encoded using C 2 to produce 8 codewords that are then to be interleaved to depth s = 3:

(c′7 and c′8 will be interleaved with the first codeword c′9 produced from the next 3 messages m 4,m 5 and m 6.) So the string of digits transmitted begins

100 110 101 010 001 011 011 000 001 011 010 001 ....

To see how decoding proceeds, suppose the first 6 digits are transmitted incorrectly; so

011 001 101 010 001 011 ...

is received. Removing the effect of the interleaving to depth s = 3 leaves the received words of length n 2 = 6:

001000, 100101 and 111011
< previous page | page_168 | next page >


page_169

< previous page | page_169 | next page >

Page 169

(notice that compared to [image] respectively, each of these has errors in the first 2 positions). C 2 detects errors in all 3 codewords (show that the syndrome wH 2 of each of these received words w is not 0, where H 2 is a parity check matrix for C 2), so all 18 digits are flagged (we shall replace them with an *). Assuming that there are no more errors, after carrying out a similar process for the subsequent received digits, [image] contain no flagged digits. Then, removing the effect of the interleaving of depth k 2 = 3 leaves three words of length n 1 = 8:

c 1 = * * * 01110
c 2 = * * * 00011
c 3 = * * * 00101.

There is exactly one way to replace each flagged digit with a 0 or a 1 to produce codewords, and the codewords produced are c 1, c 2 and c 3. Notice that each of the above has only 3 = d 1 − 1 flagged digits.

Exercises

7.2.12 Use the codes C 1 and C 2 to encode the following sets of messages by cross interleaving C 1 with C 2, with C 2 being interleaved to depth s.

(a) m 1 = 0110, m 2 = 1011, m 3 = 1111, s = 2

(b) m 1 = 0110, m 2 = 1011, m 3 = 1111, s = 3

(c) m 1 = 0010, m 2 = 1111, m 3 = 1010, s = 3

(d) m 1 = 1000, m 2 = 0100, m 3 = 0010, m 4 = 0001, m 5 = 0011, m 6 = 0100, s = 3

7.2.13 The following string of digits was originally encoded by cross interleaving the codes C 1 and C 2 of Example 7.2.11, C 2 being interleaved to depth 3. Decode the following strings by finding the most likely messages m 1, m 2, and m 3.

(a) 000001001110110001000111000111000111000000000000000000...

(b) 100011001111101010011001111010100110100100011101000100....

7.2.14 Let Ci be an (ni, ki, di) linear code for 1 ≤ i ≤ 2. Suppose C 1 is cross-interleaved with C 2 to form codewords. Find how long a burst of errors the decoding algorithm described before Example 7.2.11 will correct if these codewords are

(a) interleaved to depth s, or

(b) s-frame delay interleaved

before they are transmitted.
< previous page | page_169 | next page >


page_17

< previous page | page_17 | next page >

Page 17

Thus,

If v = 111 is transmitted, we compute the probability that IMLD correctly concludes this after one transmission. First,

L(111) = {110, 101, 011, 111},

so

Exercises

1.10.2 Suppose p = .90, |M| = 2, n = 3, and C = {001, 101}, as in Exercise 1.9.5.

(a) If v = 001 is sent, find the probability that IMLD will correctly conclude this after one transmission.

(b) Repeat part (a) for v = 101.

Both answers in Exercise 1.10.2 are θp(C, v) = .900. Comparing this to the results in Example 1.10.1, we conclude that since .900 < .972, the code C = {000, 111} is better than the code C = {001, 101}, at least when judged by the third criterion in the last section. Our method provides a procedure, (although somewhat inefficient when n is large) for determining when the probability that IMLD works is high. Fortunately, most of the codes we design later on are structured so that the calculation of this probability is much easier.

Example 1.10.3 Suppose p = .90, |M| = 3, n = 4, and C = {0000, 1010, 0111}, as in Example 1.9.4. For each v in C, we compute [image] p(C, v).

(a)
< previous page | page_17 | next page >


page_170

< previous page | page_170 | next page >

Page 170

7.3 Application to compact discs

The recording of music on compact discs has taken the music-loving world by storm. The high quality reproduction from compact disc players is due in large part to the error correcting codes that are used when storing the music. Each compact disc contains a spiral track in which pits (lower levels) have been made. A laser beam following the spiral track determines where changes in height in the spiral track occur by detecting changes in the intensity of the light reflected by the compact disc. In this way, a binary string of digits is produced, each change in height corresponding to the digit 1, an absence of a change in height being a 0 digit.

During the recording, the music is sampled 44, 100 times per second, the amplitude of the sound wave at each sample being assigned a binary word of length 16. So the range of amplitudes is divided into 216 values. Recording in stereo requires 2 amplitude measurements to be taken 44, 100 times per second, one from the left and one from the right.

For encoding purposes, each binary word of length 16 corresponding to an amplitude measurement is represented by 2 field elements in GF(28); we refer to each field element as a byte. So when recording in stereo, 4 bytes m 4t,m 4t+1,m 4t+2, and m 4t+3 are produced at each ''tick" t (calling 1/44, 100th of a second a tick). Then measurements of the amplitude from 6 consecutive ticks m 24t,m 24t+1,..., m 24t+23 are grouped together to form a message Mt of length 24, each byte being in GF(28). Let C be an RS(28,5) code. Then Mt is encoded to the codeword ct using the code C 1 = C(227), the shortened Reed-Solomon code over GF(28) with (n 1,k 1,d 1) = (28,24,5) (see Example 6.2.11).

The codewords in C, thus produced are then 4-frame delay interleaved (see Table 7.2). Notice that each column in the array in Table 7.2 in this case has length n 1 = 28. Also, since the bytes of the codeword c t occur in columns t,t + 4,t+8,...,t+108, it is natural to label the bytes in c t with c 1,t,c 2,t+4,c 3,t+8, ...,c 28,t+108.

Column t of the array in Table 7.2 contains the bytes c 1,t,c 2,t,...,c 28,t (recall that ci,j is the i th byte in the codeword c j−4(i−1)), and these are now used as messages of length 28 over GF(28) that then are encoded using C 2 = C(223), the shortened Reed-Solomon code over GF(28) with (n 2,k 2,d 2) = (32,28,5).

To each codeword in C2, one further byte is added for control and display purposes, so codewords now have length 33.
< previous page | page_170 | next page >


page_171

< previous page | page_171 | next page >

Page 171

Up until now, all bytes carry information or have been added for error correction and detection purposes. However, physical limitations of the laser tracking make it desirable that changes in height in the spiral track do not occur too close together nor too far apart. It was therefore determined that in the binary representation of each codeword, between any two 1's there should be at least two and at most ten 0's. The Reed-Solomon codewords do not have this property. However there are exactly 267 binary words of length 14 that do have this property. The 256 field elements are matched to 256 of these words using a table look up, 11 of the 267 words being discarded. This process is called eight to fourteen modulation (EFM). Then, to make sure that the property holds between the words of length 14,3 further bits (0's or 1's) are added. So now, the binary representation of each codeword has length 33 × 17 = 561.

Finally, to each codeword, a binary word of length 27 is added for synchronization purposes, which also has the above property. Therefore in total, audio information from 6 consecutive ticks is initially stored as a binary vector of length 24 × 8 = 196, and after all the processes are complete, appear on the compact disc as a binary word of length 588.

It remains to discuss the decoding. First, all extra manipulation such as the EFM (see also the remark at the end of this section) is undone to leave ''received" words which are hopefully codewords in C 1.C 2 is used to correct all single errors. However if more than one error is detected then all bytes in the received word are flagged (see Section 7.2 on cross-interleaving). The effect of the 4-frame delay interleaving is then removed. Finally C 1 is used to correct up to 4 erasures (recall C 1 has distance 5), treating all flagged bytes as erasures, and all unflagged bytes as being correct.

How good is this decoding? First, notice that the only way that the decoding using C 2 can go wrong is if the received word is within distance 1 of a C 2 codeword that is not the right codeword. There are very few error patterns that will do this! There are (2r)k = (28)28 = 2224 codewords in C 2, one of which is the right codeword. Each of the remaining 2224 − 1 codewords is within distance 1 of 1+32(28 − 1) words of length 32. So of all (28)32 binary error patterns that might be added to a C 2 codeword, only (2224 − 1)(1+32(28 − 1)) of them result in a word within distance 1 of a different C 2 codeword; that is about 1 in 219 of them. This correction of single error by C 2 is designed to cope with small random errors caused by inaccuracies in the coating and cutting of the compact discs.

Secondly, after the effect of the 4-frame delay interleaving is removed, a received word will be decoded to the correct codeword in C 1 if it contains at most 4 flagged digits (and assuming that C 2 detects all errors, which we just saw is very likely). But for a single burst to affect 5 digits of a C 1 codeword, it would have to affect 17 columns of the array in Table 7.2, or more precisely, at least 15 × 32+2+2 bytes (if 2 bytes are altered in the first or seventeenth columns, then all bytes in that column are flagged by C 2). Since each column of Ta-
< previous page | page_171 | next page >


page_172

< previous page | page_172 | next page >

Page 172

ble 7.2 is represented by a word of length 588 on the compact disc, all bursts of length (15 × 588)+(3 × 17) = 8871 (or if you prefer, all bursts affecting (15 × 24 × 8)+(3 × 8) = 2904 audio bits) are decoded correctly. This burst length corresponds to approximately 2.5mm of track length on the compact disc.

Remark Here we have presented the most important aspects of the encoding process. In practice, several other interleaving operations are incorporated into the encoding.

For example, the bytes in the odd numbered positions in the codewords in C 2 are all moved along n 2 = 32 positions, so they are mixed with the bytes in the even numbered positions of the following codeword. This is done to improve the chances of correcting single errors with C 2, since now 2 consecutive errors affect different codewords.

Also, the bytes within codewords in C 1 are reorganized. Such a codeword contains information from 6 consecutive ticks from the left and right, say L 1,...,L 6 and R 1,...,R 6 as well as two parity symbols Q 1 and Q 2 added when encoding with C 1. These are arranged in the following order:

L 1 L 3 L 5 R 1 R 3 R 5 Q 1 Q 2 L 2 L 4 L 6 R 2 R 4 R 6.

The point of this is that if several consecutive bytes are still flagged after the decoding process, they can be treated as unreliable information. In such a case, an unreliable value say Li can be replaced by an amplitude found by interpolating the (hopefully) reliable values of L i−1 and L i+1. For example, if the values L3, L5, R1, R3, R5, Q1 and Q 2 are all still flagged, L 3 can be found by ''averaging" the amplitudes of the reliable values of L 2 and L 4, and so on.
< previous page | page_172 | next page >


page_173

< previous page | page_173 | next page >

Page 173

Chapter 8
Convolutional Codes

8.1 Shift registers and polynomials

One reason cyclic codes are so useful is that polynomial encoding and decoding can be implemented easily and efficiently by hardware devices known as shift registers. Briefly, these devices consist of n registers (or delay elements) and a ''clock" which controls the movement or shifting of the data contained in the registers. After each clock "tick," the new contents of the registers are combined (binary addition) to form the output. In Figure 8.1, the squares denote registers; arrows indicate the flow of data, and ⊕ means binary addition.

Example 8.1.1 In the shift register of Figure 8.1, we have four registers X 0, X 1, X 2, X 3 each containing binary digits. As the arrows indicate, the output at each clock tick is formed by adding the contents of the registers X 0, X 1, and X 3. Suppose registers X 0, X 1, X 2, and X 3 contain 1, 1, 0, and 1 respectively. If the next input digit is 0, then at the next clock tick, the input digit is "shifted" into X0 and at the same time the contents of each register is shifted into the next. The new contents of X0, X1, X2, X3 will be 0,1,1,0, and the output digit will be 0+1+0 = 1.

Suppose we have an input sequence a0, a1,..., etc. then we can keep track of the input, output and contents of the registers at each clock tick by means of a table.

Example 8.1.2 Consider the 4-stage shift register in Figure 8.1. Assume the contents of the registers is initially (0,0,0,0) and the input stream a0, a1, a2,...a6 is

Figure 8.1: A shift register.
< previous page | page_173 | next page >


page_174

< previous page | page_174 | next page >

Page 174

1010000. The contents of the registers and outputs are summarized in the following table.

time | input | X0 X1 X2 X3 | output = X0+X1+X3
−1 | —— | 0 0 0 0 | ——
0 | 1 | 1 0 0 0 | 1
1 | 0 | 0 1 0 0 | 1
2 | 1 | 1 0 1 0 | 1
3 | 0 | 0 1 0 1 | 0
4 | 0 | 0 0 1 0 | 0
5 | 0 | 0 0 0 1 | 1
6 | 0 | 0 0 0 1 | 0

Thus the shift register outputs 1110010 for input 1010000, where the initial state of the registers was 0000.

In general, an s-stage shift register is a shift register with s registers. The output of an s-stage shift register is a linear combination of the contents of the registers and can be described using coefficients g 0,g 1,...,g s−1, with gi ∈ K = {0, 1}; that is, ct = g 0 X 0(t) +...+ g s−1 X s−1(t) where ct is the output at time t and Xi(t) is the value of the contents of register Xi at time t.

The action of these devices can be described in terms of polynomials. If g 0, g 1,...,g s−1 are the coefficients of the s-stage shift register then g(x) = g 0+g 1 x+...+g s−1 x n−1 is the polynomial corresponding to this shift register; this polynomial is the generator of the shift register. For instance g(x) = 1+x + x 3 is the generator of the 4-stage shift register in Figure 8.1. If we represent the input and output sequences by polynomials a(x) and c(x), then we claim that for input sequences a(x), the shift register with polynomial g(x) will output c(x) = a(x)g(x).

Example 8.1.3 Let g(x) = 1+x+x3, the polynomial corresponding to the shift register in Figure 8.1. The input sequence 1010000 corresponds to a(x) = 1+x 2. Assuming the 4 registers all contain 0, then from Example 8.1.2 we know that the output sequence of the device will be 1110010, or c(x) = 1+x+x 2+x 5. But,

Example 8.1.4 Let g(x) = 1+x + x 3 be the polynomial associated with the shift register of Figure 8.1. The following table gives us the output sequence for the arbitrary input sequence a 0,a 1,a 2,a 3,0,0,0.
< previous page | page_174 | next page >


page_175

< previous page | page_175 | next page >

Page 175

time | input | X 0 | X 1 | X 2 | X 3 | output = X 0+X 1+X 3
−1 | —— | 0 | 0 | 0 | 0 | ——
0 | a 0 | a 0 | 0 | 0 | 0 | a 0
1 | a 1 | a 1 | a 0 | 0 | 0 | a 1+a 0
2 | a 2 | a 2 | a 1 | a 0 | 0 | a2+a1
3 | a 3 | a 3 | a 2 | a 1 | a 0 | a3+a2+a 0
4 | 0 | 0 | a 3 | a 2 | a 1 | a 3+a 1
5 | 0 | 0 | 0 | a 3 | a 2 | a 2
6 | 0 | 0 | 0 | 0 | a 3 | a 3

Clearly,

and the coefficients of c(x) correspond to the output sequence of the device.

Given a fixed generator polynomial g(x) of degree n − k for a cyclic linear code one can build an n − k + 1 stage shift register with generator g(x) to implement polynomial encoding of information polynomials a(x).

Exercises

8.1.5 Draw the diagrams for the shift registers corresponding to generator polynomials g(x):

(a) 1+x

(b) 1+x 2

(c) 1+x 2+x 3

(d) 1+x 3+x 4

8.1.6 Use the shift registers constructed in Exercise 8.1.5 to compute a(x)g(x) = c(x). Compute a(x)g(x) directly and compare the results.

(a) g(x) = 1+x 2,a(x) = 1+x

(b) g(x) = 1+x 3+x 4,a(x) = 1+x 3+x 6

(c) g(x) = 1+x 2+x 3,a(x) = x + x 2

(d) g(x) = 1+x 3+x 4,a(x) = x 2+x 5+x 6

8.1.7 For the shift register in Figure 8.1, with g(x) = 1+x + x 3, compute the output sequence c 0, c 1,... for each input sequence a 0, a 1,... below. Assume registers are all initially zero.

(a) 10101000...

(b) 0011000...

(c) 1010010000...
< previous page | page_175 | next page >


page_176

< previous page | page_176 | next page >

Page 176

Now we show that, in general, shift registers accomplish polynomial multiplication. Let a(x) = a 0+a 1 x+...+a k−1 x k−1 and g(x) = g 0+g 1 x+...+ g l−1 x e−1. First recall that a(x)g(x) = c(x) means that ct, the coefficient of xt in c(x), is

ct = g 0 a t+g 1 a t−1+...+a 0 g t if t ≤ ℓ − 1

and

c t = g 0 at+g 1 a t−1+...+a t−ℓ+1gℓ−1 if t > ℓ − 1.

For convenience we assume that at = 0 if t > k − 1 = deg(a(x)).

Consider now the shift register with generator g(x). The output at time t, is the linear combination of the Xi(t):

ct = g 0 X 0(t)+...+gℓ−1 X ℓ−1(t).

At time t = 0,X 0(0) = a 0 and X 1(0) =...= X n−1(0) = 0, so c 0 = g 0 a 0.

More generally at time t, t ≤ ℓ − 1, X 0(t) = a t, X 1(t) = a t−1,...,X t(t) = a 0 and the remaining registers are all zero. Thus,

ct = g 0 a t+g 1 a t−1+...+g t a 0, for t ≤ ℓ − 1.

Finally at time t > ℓ − 1, we have

X 0(t) = a t,X 1(t) = a t−1,...,X ℓ−1(t) = a t−ℓ+1

and ct = g 0 at+g 1 a t−1+...+g ℓ−1 a t–ℓ+1, t > ℓ − 1. (We note again that we use the convention that if a 0,a 1,...,a k−1 is the input sequence then at = 0, for t ≥ k.)

Theorem 8.1.8 A shift register with generator g(x), given an input sequence a0, a1,... will output c 0,c 1,..., where c(x) = a(x)g(x), with c(x) = c 0+c 1 x+ ..., and a(x) = a 0+a 1 x+....

We see that polynomial multiplication (and hence polynomial encoding for cyclic codes) can be implemented using shift registers: the generator g(x) of the shift register is the generator polynomial of the linear cyclic code. We can also modify the shift registers to implement division of polynomials, which is of use when decoding linear cyclic codes. Polynomial division (and thus polynomial decoding for cyclic codes) can be implemented by hardware devices known as feedback shift registers (FSR). Basically an FSR is a shift register with the output fed back into the device.

(For those readers only interested in convolutional codes, the rest of this section may be omitted.)

First recall that in computing the parity check matrix H for the cyclic code with generator g(x), the ith row of H is ri ↔ ri(x), where
< previous page | page_176 | next page >


page_177

< previous page | page_177 | next page >

Page 177

Figure 8.2: A feedback shift register.

In particular

ri(x) = xri−1(x) mod g(x)

Example 8.1.9 Let g(x) = 1+x+x 3 be the generator polynomial. In the parity matrix,

But r 5 ↔ x 2+x 3 mod g(x) or r 5 = 001+110. We consider the vector 110 to be the feedback vector which is added back in to the registers if the output digit is 1. The feedback shift register in Figure 8.2 performs this operation.

We use the convention that if more than one input comes into a register the contents will be the binary sum of their values. That is, the two diagrams below represent the same thing.

At each clock tick the input and contents of the registers are shifted and the output digit ct, is added back into selected registers. Equivalently the (new) vector ct(1, 1, 0) is added to the contents of the registers.

time | input | X 0+c t | X 1+c t | X 2 | c t = output
−1 | —— | 0 | 0 | 0 | ——
0 | 1 | 1 | 0 | 0 | 0
1 | 0 | 0 | 1 | 0 | 0
2 | 0 | 0 | 0 | 1 | 0
3 | 0 | 0+1 | 0+1 | 0 | 1
4 | 0 | 0 | 1 | 1 | 0
5 | 0 | 0+1 | 0+1 | 1 | 1
6 | 0 | 0+1 | 1+1 | 1 | 1
7 | 0 | 1 | 0 | 0 | 1

In general, an s-stage FSR with feedback vector (g 0,g 1,...,g s−1) corresponds to a polynomial g(x) = g 0+g 1 x+...+g s−1 x s−1+x s of degree s. The contents of the registers at time t = deg(c(x)) will be the remainder of c(x)/g(x) and the output sequence will be the quotient a(x). However, the received word must be fed in reverse order, high order digits first. (Notice that for an FSR, the associated

< previous page | page_177 | next page >


page_178

< previous page | page_178 | next page >

Page 178

polynomial has degree s, whereas the polynomial associated with a shift register has degree s − 1; however in both cases we have s registers.)

Example 8.1.10 Let x+x 2+x 4 be the received polynomial and 0110100 be the corresponding word. Assume g(x)=1+x+x 3 and so the corresponding FSR be as in Figure 8.2.

time | input | X 0 | X 1 | X 2 | output
−1 | —— | 0 | 0 | 0 | ——
0 | 1 | 1 | 0 | 0 | 0
1 | 0 | 0 | 1 | 0 | 0
2 | 1 | 1 | 0 | 1 | 0
3 | 1 | 1+1 | 1+1 | 0 | 1
4 | 0 | 0 | 0 | 0 | 0

The remainder is 000 and the quotient x ↔ 0100000, corresponding to the output sequence in reverse order.

In general the contents of the registers at time t=n, will be the remainder of c(x) mod g(x), where c(x) represents the input sequence.

Theorem 8.1.11 Feeding c(x) = c 0+c 1 x+...+c n−1 x n−1 into a FSR corresponding to g(x) = g 0+g 1 x+...+1.x s with high order coefficients fed in first (i.e.,c n−1,c n−2,...,c 0) is equivalent to dividing c(x) by g(x). The output after n clock ticks will be the quotient (high order coefficients first) and the registers will contain the remainder (high order digits to the right).

Proof: Since the output of the sum of two input streams will be the sum of the corresponding output streams all we need to do is verify this theorem for c(x) = xl. But it is clear that the FSR corresponds to our earlier algorithm for computing xl mod g(x) (see Example 4.3.7). Thus the registers will contain the remainder. It is less obvious but not too difficult to prove that the output will be the quotient on dividing c(x) by g(x).

Exercises

8.1.12 Given the feedback shift register in Figure 8.2, with the registers initially set to zero, generate the output sequence for each of the following received words. Indicate the final state of the registers and quotient if the remainder is zero.

(a) 0011010 (b) 1010110 (c) 0010001

8.1.13 Given g(x) = 1+x+x 3, compute the syndrome polynomials for each of the received words in Exercise 8.1.12. Compare the syndrome polynomial with the corresponding final state of the registers computed in 8.1.12.

8.1.14 For each generator polynomial g(x) construct a corresponding feedback shift register. Compute the output sequence and find the final state of the registers for the given input sequence c(x).
< previous page | page_178 | next page >


page_179

< previous page | page_179 | next page >

Page 179

(a) g(x) = 1+x 2+x 3,c = 0010110

(b) g(x) = 1+x+x 2,c = 111

(c) g(x) = 1+x+x 4,c = 010000000100000

8.2 Encoding convolutional codes

Convolutional codes are extremely practical codes. They have been adopted by both NASA and the European Space Agency for ensuring that communications during space missions are reliable. In fact, they are used in conjunction with Reed-Solomon codes: each message is first encoded with a Reed-Solomon code, and the resulting codeword is then encoded with a convolutional code. In the following sections we consider the encoding and decoding of convolutional codes, then some problems that arise with these codes. We begin with a definition.

An (n, k = 1, m) (binary) convolutional code with generators g1(x),...gn(x) where gi(x) = gi0,+gi1,x+...+gi,m xm, gi ∈ K[x] is the code consisting of all codewords

c(x) = (c 1(x), c 2(x),..., cn(x))

where ci(x) = m(x)gi(x) and m(x) = m 0+m 1 x+m 2 x 2+... ∈ K[x]. (We will briefly discuss the parameter k, later, which for simplicity is set equal to one in this definition.) Of course, m(x) is the message and is encoded to c(x). Suppose that c(x) and c′(x) are codewords. Then

which is just the codeword corresponding to the message m(x)+m′(x). Therefore convolutional codes are linear codes.

Convolutional codes are different from the codes considered so far in that they are codes of infinite length, and the message also has infinite length.

Example 8.2.1 Let C 1 be the (2, 1, 3) convolutional code with g 1(x) = 1+x+x 3 and g 2(x) = 1+x 2+x 3. We use C 1 to encode the following messages.

(a) The message m(x) = 1+x 2 is encoded to

(b) The message m(x) = 1+x+x 2 x 3+ . . . = [image] is encoded to
< previous page | page_179 | next page >


page_18

< previous page | page_18 | next page >

Page 18

(b)

(c)

Examining the three probabilities, we see that the probability that IMLD will conclude correctly that 0111 was sent is not too bad. However the probability that IMLD will conclude correctly that either 0000 or 1010 was sent is horrible. Thus, at least by the third criterion in the last section, C = {0000, 1010, 0111} is not an especially good choice for a code.

Exercises

1.10.4 Suppose p = .90 and C = {000, 001, 110}, as in Exercise 1.9.6. If v = 110 is sent, find the probability that IMLD will correctly conclude this, and the probability that IMLD will incorrectly conclude that 000 was sent.

1.10.5 For each of the following codes C calculate θp(C, v) for each v in C using p = .90. (The IMLD tables for these codes were constructed in Exercise 1.9.7.)

(a) C = {101, 111, 011}

(b) C = {000, 001, 010, 011}

(c) C = {0000, 0001, 1110}

(d) C = {0000, 1001, 0110, 1111}

(e) C = {00000, 11111}

(f) C = {00000, 11100, 00111, 11011}

(g) C = {00000, 11110, 01111, 10001}

(h) C = {000000, 101010, 010101, 111111}

1.11 Error-detecting codes

We now make precise the notion of when a code C will detect errors. Recall that if v in C is sent and w in Kn is received, then u = v + w is the error pattern. Any

< previous page | page_18 | next page >


page_180

< previous page | page_180 | next page >

Page 180

Figure 8.3: Encoding the (2, 1, 3) convolutional code C 1.

Exercises

8.2.2 Encode the following messages using the (3, 1, 3) convolutional code with generators g1(x) = 1+x+x 3, g2(x) = 1+x+x 2+x 3 and g3(x) = 1+x 2+x 3.

(a) m(x) = 1+x 3

(b) m(x) = 1+x+x 3

(c) m(x) = 1+x+x+x 2+... =

8.2.3 Encode the following messages using the (2, 1, 4) convolutional code with generators g1(x) = 1+x 3+x 4 and g2(x) = 1+x+x 2+x 4.

(a) m(x) = 1+x+x 2

(b) m(x) = 1+x+x 3

(c) m(x) = 1+x 2+x 4+... =

As can easily be seen from Theorem 8.1.8, an alternate description of convolutional codes can be given in terms of shift registers: ci(x) is the output from the shift register with generator gi(x) when m(x) is the input.

Example 8.2.4 Let's consider the convolutional code C 1 in Example 8.2.1. We can describe the code with a shift register with the 2 generators g1(x) = 1+x+x 3 and g2(x) = 1+x 2+x 3 (see Figure 8.3).

Using this description, if we encode m(x) = 1+x 2 ↔ 10100... then it is clear that c 1 = 11100100... as was calculated in Example 8.1.2 (and this agrees with the calculation in Example 8.2.1), and similarly c 2 can be shown to be 10011100....

Of course c(x) can be made into a single stream of digits, instead of the n streams we have been describing, by interleaving c 1(x),...,cn(x). For the rest of this chapter we will display c(x) in this interleaved form, so the output consists of the coefficients of x 0 in c 1(x),..., cn(x) followed by the coefficients of x,x 2,.... When displaying the interleaved form of c ↔ c(x), the n digits consisting of the coefficients of xi, for i ≥ 0, will be grouped together.

Example 8.2.5 The interleaved representation of c(x) in Example 8.2.1(a) is

c = 11 10 1001 01 11 00 00...,
< previous page | page_180 | next page >


page_181

< previous page | page_181 | next page >

Page 181

and in Example 8.2.1(b) is

c = 11 01 00 11 11 11....

Exercises

8.2.6 For the convolutional codes in Exercises 8.2.2 and 8.2.3, construct the relevant shift register which can be used to encode the code. Then check your answers to those exercises by using the shift registers to encode the given messages. Finally, represent each codeword in its interleaved form.

Considering an (n, 1, m) convolutional code as being encoded with shift registers, when 1 message digit is moved into the shift register, n code digits are produced. Therefore each message digit in an (n, k = 1, m) binary convolutional code in effect produces n code digits, one in each of c 1(x),..., cn(x)), so the rate of such a convolutional code is defined to be 1/n (recall that the rate of a code measures the fraction of information that each code digit carries). One might then ask if it is possible to design convolutional codes with rates other than 1/n, and in particular rates higher than 1/2.

The obvious way to do this is to move more than one, say k, message digits into the shift register before calculating the next code digits, thus producing a code of rate k/n. This is, in fact the definition of the parameter k for an (n, k, m) convolutional code. Notice that if we do this then each message digit will appear in the registers Xi, Xi+k, Xi+2k,..., for some i, 0 ≤ i < k. Therefore, rather than moving k message digits at a time into the shift register, we could equivalently divide the shift register into k shift registers X 0 Xk X 2k..., X 1 Xk+1 X2k+1..., and so on. Correspondingly, the message is divided into k streams, each stream being fed into one of the k shift registers. The one complication is that now contents from registers in different shift registers may be combined in a single generator. This last description is the method of encoding that is used in practice. The following example makes all this clear.

Example 8.2.7 Use the (3, 2, 3) convolutional code C with generators g1(x) = 1+x 3, g2(x) = 1+x+x 3 and g3(x) = x+x 2+x 3 to encode the message m = 100101110000....

The first interpretation of k = 2 is to encode m using the single shift register in Figure 8.4 and moving k = 2 message digits in to the shift register at each tick. Then the contents of the registers and the outputs are summarized in the following table.
< previous page | page_181 | next page >


page_182

< previous page | page_182 | next page >

Page 182

Figure 8.4: Encoding a (3, 2, 3) convolutional code.

Figure 8.5: Encoding a (3, 2, 3) convolutional code.

time | input | X 0 X 1 X 2 X 3 | output c 1 c 2 c 3
−1 | − | 0 0 0 0 | −
0 | 01 | 0 1 0 0 | 0 1 1
1 | 10 | 1 0 0 1 | 0 0 1
2 | 10 | 1 0 1 0 | 1 1 1
3 | 11 | 1 1 1 0 | 1 0 0
4 | 00 | 0 0 1 1 | 1 1 0
5 | 00 | 0 0 0 0 | 0 0 0

So m is encoded to the codeword (in interleaved form)

c = 011 001 111 100 110 000 ...

The second interpretation of k = 2 is to notice that the first, third, fifth, ... message digits fed into this shift register only ever appear in X 0 and X 2, and the second, fourth, sixth,... message digits fed into the shift register only every appear in X 1 and X 3. Therefore we can split the message and the registers into k = 2 parts as Figure 8.5 suggests.
< previous page | page_182 | next page >


page_183

< previous page | page_183 | next page >

Page 183

Exercises

8.2.8 Encode the following messages using the (3,2,4) convolutional code with generators g1(x) = 1 + x 3, g2(x) = x + x 4 and g3(x) = 1 + x + x 2 + x 3 + x 4. Use both techniques of encoding described above.

(a) m(x) = 1 + x + x 3 + x 4 + x 5

(b) m(x) = 1 + x 3+x 5+x 7+x 8

(c) m(x) = 1 + x + x 2+x 3

The rest of the chapter mainly deals with rate r = ½, (2, 1, m) binary convolutional codes. The results and techniques presented can be generalized to (n,k,m) convolutional codes, but the main ideas can all be found in this notationally simpler setting. For those interested in convolutional codes with k > 1, we have included some exercises that indicate how to generalize the material presented for k = 1.

It is worth noting that NASA's Mars Global Surveyor uses a convolutional code with r = ½ and m = 7 when sending information to Earth from Mars, and the Mars Pathfinder can choose a convolutional code with (r,m) = (½, 7) or (1/6, 15). At the time of printing, photographs taken by these NASA missions were displayed on the internet at ''http://www.msss.com".

Finally, there is another way of viewing the encoding of convolutional codes. Recall that a (2, 1, m) convolutional code can be encoded using a shift register containing m + 1 registers. At each tick, the contents of the first m registers is called the state of the shift register. The zero state is the state when each of the first m registers contains 0. If the shift register is currently in state s 0,s 1,...,s m−1 then at the next tick, it will be either in state 0, s 0, s 1,...,s m−2 or in state 1, s 0, s 1,...,s m−2, depending on whether the message digit shifted in to register X 0 was a 0 or a 1 respectively. Also, if we know the current state s 0, s 1,...,s m−1 and the previous state s 1,s 2,...,sm, then we know the current contents of ALL of the registers; therefore we know the current output. This information is often presented graphically: the state diagram of a (2, 1, m) convolutional code is a directed graph in which the vertices, or states, are all binary words of length m, and for each state s = s 1, s 2,...,sm there is an edge directed from s to state 0, s 1,s 2,...,s m−1 that is labeled with the output when the registers X 0, X 1,...,Xm contain 0, s 1,...,sm respectively and there is an edge directed from s to state 1, s 1,s 2,...,s m−1 that is labeled with the output when the registers X 0, X 1,...,Xm contain 1, s 1,...,sm respectively.

The information in the state diagram of a (2, 1, m) convolutional code can also be represented in tabular form. Each row of the table lists the current state (that is, the contents of registers X 0, X 1,...,X m−1) and the corresponding output, which of course depends on whether Xm = 0 or Xm = 1.
< previous page | page_183 | next page >


page_184

< previous page | page_184 | next page >

Page 184

Figure 8.6: The state diagram for C 1.

Example 8.2.9 Let C 1 be the (2, 1, 3) convolutional code with generators g1(x) = 1 + x + x 3 and g2(x) = 1 + x 2 + x 3 (see Examples 8.2.1 and 8.2.4). The states are all binary words of length m = 3 : 000, 100, 010, 001, 110, 101, 011, 111. Considering the state s = s 1 s 2 s 3 = 011 for example, there is a directed edge from s to the state 0s1 s2 = 001, and a directed edge from s to 1s 1 s 2 = 101. The directed edge from 011 to 001 is labeled with the output when X 0 X 1 X 2 X 3 = 0011, namely 10, and the edge directed from 011 to 101 is labeled with the output when X 0 X 1 X 2 X 3 = 1011, namely 01. If we do this for every state then we obtain the state diagram in Figure 8.6.

This state diagram can also be represented by the following table.

State | Output
X 0 X 1 X 2 | X 3 = 0 | X 3 = 1
000 | 00 | 11
100 | 11 | 00
010 | 10 | 01
110 | 01 | 10
001 | 01 | 10
101 | 10 | 01
011 | 11 | 00
111 | 00 | 11

Recall that initially the contents of each register is set equal to 0, so the initial state of the shift register is X 0 X 1...X m−1 = 00...0. As each message digit is fed into the shift register, the shift register moves to a different state and each generator outputs a code digit. In the state diagram, this obviously corresponds to moving from state to adjacent state following directed edges, the outputs being the labels on the directed edges. In this way, a codeword naturally corresponds to a (directed) walk in the state diagram that begins at the zero state and moves along directed edges to adjacent states. Notice that at each tick, the message digit being moved in to the shift register is the first digit in the state in the state diagram

< previous page | page_184 | next page >


page_185

< previous page | page_185 | next page >

Page 185

to which the shift register moves. Therefore it is also easy to recover the message corresponding to any codeword.

Example 8.2.10 Continuing Examples 8.2.1 and 8.2.9, the message m(x) = 1 + x 2 ↔ 10100...corresponds to the walk starting at state 000, then moving to states 100, 010, 101, 010, 001, 000, 000,..., in turn. The labels on these directed edges are 11, 10, 10, 01, 11, 00,..., respectively, and this is precisely the codeword to which m(x) is encoded (in interleaved form; see Example 8.2.5).

Also, given a codeword, we can easily recover the corresponding message. Consider the codeword

c = 00 11 01 11 01 01 01 11 00....

The walk in the state diagram that produces c proceeds through the states

000, 000, 100, 110, 011, 101, 010, 001, 000, 000...

(of course, all codewords start at the zero state 000). Since, at each tick, the message digit is the first digit in the state to which the register moves, the message corresponding to c is produced by writing down the first digit in each of the states in the walk other than the initial state:

m=0 1 1 0 1 0 0 0 0 ...

Exercises

8.2.11 (a) Find the state diagram, and its representation in tabular form, for the (2, 1, 2) convolutional code with generators g 1(x) = 1+x 2 and g2(x) = 1+x + x 2.

(b) Use the state diagram to encode the following messages:

(i) m(x) = 1+x 2

(ii) m(x) = 1+x + x 2

(c) Use the state diagram to find the message corresponding to the following codewords:

(i) 11 01 00 01 11 00 ...

(ii) 00 11 10 01 01 10 00 ...

8.2.12 (a) Find the state diagram, and its representation in tabular form, for the (2, 1, 3) convolutional code with generators g1(x) = 1 + x + x 2 + x 3 and g2(x) = 1 + x 2 + x 3.

(b) Use the state diagram to encode the following messages.

(i) m(x) = 1 + x 3

(ii) m(x) = 1 + x + x 3
< previous page | page_185 | next page >


page_186

< previous page | page_186 | next page >

Page 186

(iii) m(x) = 1 + x + x 2 + ...=

(c) Use the state diagram to find the message corresponding to the following partial codewords.

(i) 11 10 00 01 00 10 10...

(ii) 00 11 01 10 01 10 00 ...

8.2.13 (a) Find the state diagram, and its representation in tabular form, for the (2, 1, 4) convolutional code with generators g1(x) = 1 + x 3 + x 4 and g 2(x) = 1 + x + x 2 + x 4.

(b) Use the state diagram to encode the following messages.

(i) m(x) = 1 + x + x 2

(ii) m(x) = 1 + x + x 3

(iii) m(x) = 1 + x 2 + x 4 + ... =

Compare your answers to those of Exercise 8.2.3.

8.2.14 For 1 ≤ k ≤ m/2, we can similarly define a state diagram for (n, k, m) convolutional codes. The states are all binary words of length m + 1 − k and for each state s = sk, sk + 1,...,sm and for each binary word u of length k there is an edge directed from state s to state u, sk, sk + 1,..., sm−k that is labeled with the output when the registers X0, X1,...,Xm contain u, sk, sk+1,..., sm (for k > 1, here we are using the encoding description of shifting k message digits into a single shift register at each tick).

Find the state diagram for the (n, k, m) convolutional codes with the following generators.

(a) g 1(x) = 1 + x + x 3, g 2(x) = 1 + x + x 2 + x 3, and g 3(x) = 1 + x 2 + x 3, with k = 1

(b) g 1(x) = 1 + x 3, g 2(x) = 1 + x + x 3, and g3(x) = x + x 2 + x 3, with k = 2

(c) g 1(x) = 1 + x 3, g 2(x) = x + x 4, and g 3(x) = 1 + x + x 2 + x 3 + x 4, with k = 2

8.3 Decoding convolutional codes

Clearly, decoding convolutional codes is going to be somewhat different from the decoding of other codes because each codeword has infinite length. To avoid storage problems, decoding must begin before the entire codeword is received, so it is natural to consider how long we should wait before beginning to decode. For example, consider C 1, the (2, 1, 3) convolutional code with generators g 1(x) = 1 + x + x 3 and g 2(x) = 1 + x 2 + x 3 (the state diagram for this code in Figure 8.6 will be useful in the following discussion). Suppose that the received word is w(x) = 1 + x ↔ 11 00 00 00 ... = w. We know that codewords correspond to directed walks in the state diagram that start at state 000, but clearly there is no directed walk that would give an output of w. Therefore we are faced with finding

< previous page | page_186 | next page >


page_187

< previous page | page_187 | next page >

Page 187

a codeword that ''most closely" fits w; that is, a directed walk in the state diagram with an output that is close to w.

If we know all of w, then the directed walk that never leaves state 000 produces the codeword c 1 = 00 00 00 ... that is distance 2 from w. It is not hard to check that every other walk produces an output that differs from w in more than 2 places, so c 1 is the "closest codeword." So we decode w to the message m = 000....

However, suppose that our storage capabilities are so limited that at each tick we have to decode a message digit. At the first tick, we start at state 000 and we see the digits 11 in w. Our best choice is to move to state 100, because 11 is the label on the directed edge from state 000 to state 100, so is an output that agrees with the received digits of w. Therefore we decode the first message digit as 1. At the second tick, we are in state 100, we see the digits 00 of w and so we are in a quandary: moving to adjacent state 010 or 110 produces an output of 10 or 01 respectively, both of which are distance 1 from the received digits. We are forced to make an arbitrary decision, knowing that errors have occurred during transmission that we have been unable to correct. So we have decoded w to either c 2 = 11 10 ... or to c′2 = 11 01 ..., with the most likely message being m = 1*... (we write * whenever we face an arbitrary decision between decoding a 0 or a 1). Notice that if we decode *, then the next current state must also be arbitrarily chosen from the two states adjacent to the current state.

Consider one further possibility, where we can store two ticks worth of information before decoding. So now we can begin by considering all walks of length 2 from the zero state and compare their labels with the first 2 ticks of w, namely 11 00, to obtain the information in Figure 8.7. Two of the walks are closest to 11

Walk | Output | Distance from 11 00
000, 000, 000 | 00 00 | 2
000, 000, 100 | 00 11 | 4
000, 100, 010 | 11 10 | 1
000, 100, 110 | 11 01 | 1

Figure 8.7: Information for the first decoding decision.

00, the part of w that we have seen so far. However this is not a problem, because both of these walks agree that we should first move to state 100. These two walks only disagree as to where to proceed after state 100, but that decision need not yet be made; we make that decision after receiving another two digits of w. Therefore we make the decoding decision to move to state 100 and decode the first message digit as 1. Now we use the second and third ticks of information of w, namely 00 00, to make the next decoding decision. We consider the distance from 00 00 to the output of each walk of length 2 from the current state 100 (see Figure 8.8). In this case there is a unique closest walk to the piece of w we are currently considering. That walk is 100, 110, 111, so we make the decoding decision to move
< previous page | page_187 | next page >


page_188

< previous page | page_188 | next page >

Page 188

Walk | Output | Distance from 00 00
100, 010, 001 | 10 01 | 2
100, 010, 101 | 01 11 | 2
100, 110, 011 | 01 11 | 3
100, 110, 111 | 01 00 | 1

Figure 8.8: Information for the second decoding decision.

to state 110 and decode the second message digit to 1.

Exercises

8.3.1 Let C be the (2, 1, 3) convolutional code with generators g1(x) = 1 + x + x 2 + x 3 and g2(x) = 1 + x 2 + x 3 (the state diagram for C is constructed in Exercise 8.2.12). By forming tables similar to those in Figures 8.7 and 8.8, decode the first 4 message digits of the received word w = 11 00 00 00 ... ↔ 1 + x = w(x) by waiting for

(a) 2 ticks before decoding,

(b) 3 ticks before decoding,

(c) 4 ticks before decoding.

Decode the ''message digit" * if two closest walks disagree as to which state to move. If an * is decoded, assume that the message digit 0 is decoded to determine the next current state.

Notice that if we decide to wait τ steps before decoding, then a decoding decision looks at all walks of length τ from the current state, compares each such walk to the τ ticks of information of the received word currently in our possession, then moves to the next state in all walks that most closely agree with w. Another tick of w is received before another step is taken. Also, if two closest walks to w disagree as to which state to move to (as was the case at the second tick when we chose τ = 1), then we could arbitrarily select one of the states. Call this decoding algorithm the exhaustive decoding algorithm for convolutional codes (because all walks of length τ from the current state are considered for each message digit to be decoded), and call τ the window size (since τ is the amount of w we "see" when making each decoding decision).

Clearly the amount of time we wait before making a decoding decision is affecting our answer to what is the closest codeword. The problem now is to see if there is a happy medium, between making a decoding decision at every tick and decoding after all of the received word has been seen, that allows all error patterns of a certain type to be corrected. But this raises another question: which error patterns can be corrected? We will obtain several answers to this question, then in each case address the problem of how long to wait before decoding.
< previous page | page_188 | next page >


page_189

< previous page | page_189 | next page >

Page 189

First, however, we need to consider yet another problem! Consider the (2, 1, 3) convolutional code with generators g1(x) = 1 + x 3 and g2(x) = 1 + x + x 2. The state diagram for this code is in Figure 8.9. Suppose that the codeword transmit-

Figure 8.9: The state diagram of a catastrophic convolutional code.

ted is the zero codeword, with corresponding message m = 000..., and that the received word is

w = 11 10 00 00... ↔ 1 + x + x 2 = w(x).

Decoding w is simple because it is a codeword, as can be seen from the state diagram, following the walk through the states 000, 100, 110, 011, 101, 110,.... In this case we presume that no errors occurred during transmission, and so we assume that the most likely message is m = 1 1 0 1 1 0... ↔ [image]. This is a disastrous situation because in fact what happened was that the first three digits were transmitted incorrectly (we were assuming that c = 00 00... was sent) and this led us to make infinitely many errors in decoding (because we decoded m = 110110... instead of m = 000000...). Fortunately it is easy to see what the problem is: the state diagram has a cycle, other than the loop on the zero state, in which every directed edge is labeled with the zero output. Whenever such a cycle occurs, this problem of finitely many errors during transmission causing infinitely many decoding errors can occur. Define the weight of a walk (or cycle) in the state diagram to be the weight of the outputs on the directed edges in the walk. Then a convolutional code is called catastrophic if its state diagram contains a zero weight cycle different from the loop on the zero state. (The loop on the zero state in any convolutional code has weight 0. Why?) It is not hard to prove that a (2, 1, m) convolutional code is catastrophic precisely when the gcd(g1(x),g2(x)) ≠ 1. Notice that in the convolutional code we are considering, g1(x) = 1 + x 3 = (1 + x)(1 + x + x 2), so gcd(g1(x),g2(x)) = 1 + x + x 2 ≠ 1.
< previous page | page_189 | next page >


page_19

< previous page | page_19 | next page >

Page 19

word u in Kn can occur as an error pattern, and we wish to know which error patterns C will detect.

We say that code C detects the error pattern u if and only if v + u is not a codeword, for every v in C. In other words, u is detected if for any transmitted codeword v, the decoder, upon receiving v + u can recognize that it is not a codeword and hence that some error has occurred.

Example 1.11.1 Let C = {001, 101, 110}. For the error pattern u = 010, we calculate v + 010 for all v in C:

001 + 010 = 011, 101 + 010 = 111, 110 + 010 = 100.

None of the three words 011, 111, or 100 is in C, so C detects the error pattern 010. On the other hand, for the error pattern u = 100 we find

001 + 100 = 101, 101 + 100 = 001, 110 + 100 = 010.

Since at least one of these sums is in C, C does not detect the error pattern 100.

Exercises

1.11.2 Let C = {001, 101, 110}. Determine whether C will detect the error patterns (a) 011, (b) 001, and (c) 000.

1.11.3 For each of the following codes C determine whether or not C detects u:

(a) C = {00000, 10101, 00111, 11100}

(i) u = 10101 (ii) u = 01010 (iii) u = 11011

(b) C = {1101, 0110, 1100}

(i) u = 0010 (ii) u = 0011 (iii) u = 1010

(c) C = {1000, 0100, 0010, 0001}

(i) u = 1001 (ii) u = 1110 (iii) u = 0110

1.11.4 Which error patterns will the code C = Kn detect?

1.11.5

(a) Let C be a code which contains the zero word as a codeword. Prove that if the error pattern u is a codeword, then C will not detect u.

(b) Prove that no code will detect the zero error pattern u = 0.

The table constructed for IMLD can be used to determine which error patterns a code C will detect. The first column lists every word in Kn. Hence the first column can be reinterpreted as all possible error patterns, in which case the ''error pattern" columns in the IMLD table then contain the sums v + u, for all v in C. If in any particular row none of these sums are codewords in C, then C detects the error pattern in the first column of that row.
< previous page | page_19 | next page >


page_194

< previous page | page_194 | next page >

Page 194

these walks are extended at tick t + 1.

Algorithm 8.4.1 (truncated Viterbi decoding of (n, 1, m) convolutional codes with window size τ) Let w 0 w 1 ... be the received word.

1. (Initialization) If t = 0 then define

2. (Distance calculation) For t > 0 and for each state s = s 0,s 1,...,s m−1, define

where d i(s) is the distance between wt−1 and the output on the directed edge from s 1, s 2,..., s m−1, i to s.

3. (Walk calculation)

(a) If d(s 1,...,sm−1, i;t − 1) + d i(s) < d(s 1,...,s m−1, j;t − 1) + dj(s), {i, j} = {0, 1}, then form W(s;t) from W(s 1,...,s m−1, i;t − 1) by adding the leftmost digit of s to the left of W(s 1,...,s m−1, i;t − 1) and then deleting the rightmost digit.

(b) If d(s 1,..., s m−1, 0;t − 1) + d 0(s) = d(s 1,..., sm, 1;t − 1)+d 1(s) form W(s, t) from W(s 1,...,s m−1, 0;t −1) by adding the leftmost digit of s to the left of W(s 1,..., s m−1, 0;t − 1), replacing each digit that disagrees with W(s 1,..., s m−1, 1;t − 1) with *, and then deleting the rightmost digit.

4. (Decoding) For t ≥ τ, let S(t) = {s | d(s;t) ≤ d(s′;t) for all states s′}. If the rightmost digit in W(s;t) is the same, say i, for all s ∈ S(t) then decode the message digit i; otherwise decode the message digit *.

Remark Notice that the leftmost m digits in W(s;t) necessarily equal s, so do not need to be stored. In Exercise 8.4.6 a generalization of Algorithm 8.4.1 is presented that decodes (n, k, m) convolutional codes.

Example 8.4.2 Consider the code C 1 with g1(x) = 1 + x + x 3 and g2(x) = 1 + x 2 + x 3. Let w = w 0 w 1 w 2... = 11 00 00... ↔ 1 + x be the received word. We considered this example in some detail in Section 8.3. Choose a window size of τ = τ(2) = 7 (see Example 8.3.5). Recall that the state diagram for C 1 is Figure 8.10.

t = 0 Define W(s;0) = s**** for all states s, define d(000;0) = 0 and d(s′;0) = ∞ for all states s′ other than the zero state.
< previous page | page_194 | next page >


page_195

< previous page | page_195 | next page >

Page 195

Figure 8.10: The state diagram of C 1.

t = 1, w t-1 = w 0 = 11. From step 2 of Algorithm 8.4.1, we consider each state in turn.

s = 000:

(The fact that d 0(000) = 2 follows from observing that the output on the directed edge from state 000 to 000 is 00, which differs from w 0 = 11 in 2 positions. Similarly d 1(000) = 0 since the output on the directed edge from state 001 to 000 is 11 which differs from w 0 in no positions.) Using the notation of step 3a of Algorithm 8.4.1, this minimum is achieved when i = 0, so we form W(000;1) from W(000,0) by adding the leftmost digit of state 000 to W(000;0) = 000**** and then deleting the rightmost digit; therefore W(000; 1) = 0000***.

s = 100:

Again the minimum is achieved when i = 0, so form W(100; 1) from W(000; 0) by adding the leftmost digit of s = 100 to W(000; 0) then deleting the rightmost digit; so W(100; 1) = 1000***.

s = 010:

In this case we use step 3b, since the minimum is achieved by both terms. We

< previous page | page_195 | next page >


page_196

< previous page | page_196 | next page >

Page 196

have

The fourth ''digit" of W(010; 1) is an * because W(100; 0) and W(101; 0) disagree in this position.

Similarly, we can calculate W(s;t) and d(s;t) for the remaining states. Altogether we have calculated the following.

State |  |
s | t = 0 | t = 1
000 | 0, 000**** | 2, 0000***
100 | ∞, 100**** | 0, 1000***
010 | ∞, 010**** | ∞, 010****
110 | ∞, 110**** | ∞, 110****
001 | ∞, 001**** | ∞, 001****
101 | ∞, 101**** | ∞, 101****
011 | ∞, 011**** | ∞, 011****
111 | ∞, 111**** | ∞, 111****

(Each entry in the above table is: d(s;t), W(s;t).)

Notice that the tabular representation of the state diagram lists beside each state s, the output on the edges directed in to s in the state diagram. These are precisely the outputs needed in the calculation of d 0(s) and d 1(s), so the tabular form is extremely useful here. We shall include it in the following tables.

Continuing the decoding process for t = 2 and t = 3, we obtain the following (notice now w 1 = 00 and w 2 = 00).

State | Output |  |  |
s = X 0 X 1 X 2 | X 3 = 0 | X 3 = 1 | t = 2 | t = 3
000 | 00 | 11 | 2,00000** | 2, 000000*
100 | 11 | 00 | 4,10000** | 4, 100000*
010 | 10 | 01 | 1,01000** | 5, 010000*
110 | 01 | 10 | 1,11000** | 5, 110000*
001 | 01 | 10 | ∞,001**** | 2, 001000*
101 | 10 | 01 | ∞,101**** | 2, 101000*
011 | 11 | 00 | ∞,011**** | 3, 011000*
111 | 00 | 11 | ∞,111**** | 1, 111000*

Notice that we have reached t = 3 = m. At this point d(s;t) < ∞ for all states s. This represents the fact that there is a walk of length m from the zero state to each other state. Until this point, when calculating the minimum value of the set in step 2 of Algorithm 8.4.1, one of the two values was ∞. For t > m, this is no longer the case.
< previous page | page_196 | next page >


page_197

< previous page | page_197 | next page >

Page 197

t = 4 w3 = 00. Consider each state in turn.

s = 000:

with the minimum being achieved when i = 0; so W(000; 4) is produced from W(000;3). Therefore W(000; 4) = 0000000.

s = 100:

with the minimum being achieved when i = 1; so W(100; 4) is produced from W(001; 3). Therefore W(100; 4) = 1001000.

s = 010:

(where d 0 is the distance from w 3 to the output on the directed edge from state 100 to state 010, so d 0 = 1, and d 1 is the distance from w 3 to the output on the directed edge from 101 to 100, so d 1 = 1). Therefore W(010; 4) is produced from W(101; 3); so W(0101; 4) = 0101000.

Proceeding similarly for each other state we get the following.

State | Output |  |
s | X 3 = 0 | X 3 = 1 | t = 4
000 | 00 | 11 | 2,0000000
100 | 11 | 00 | 2,1001000
010 | 10 | 01 | 3,0101000
110 | 01 | 10 | 3,1101000
001 | 01 | 10 | 4,0011000
101 | 10 | 01 | 4,1011000
011 | 11 | 00 | 1,0111000
111 | 00 | 11 | 3,1111000

t = 5 w 4 = 00. Consider each state in turn.

s = 000:
< previous page | page_197 | next page >


page_198

< previous page | page_198 | next page >

Page 198

and W(000; 5) = 0000000.

s = 100:

In this case, d(000; 4)+d 0 = d(001; 4)+d 1, so from step 3b, W(100; 5) is produced from both W(000; 4) and W(001; 4) by putting an * whenever they disagree, adding the leftmost digit of 100 to the left and deleting the rightmost digit. Since W(000; 4) = 0000000 and W(001; 4) = 0011000, we get W(100; 5) = 100**00.

Proceeding similarly for the remaining states and then for t = 6 and 7 produces the following information.

State | Output |  |
s | X 3 = 0 | X 3 = 1 | t = 5 | t = 6 | t = 7
000 | 00 | 11 | 2,0000000 | 2,0000000 | 2,0000000
100 | 11 | 00 | 4,100**00 | 2,1001110 | 4,100****
010 | 10 | 01 | 3,0100100 | 3,0101110 | 3,0100111
110 | 01 | 10 | 3,1100100 | 3,1101110 | 3,1100111
001 | 01 | 10 | 2,0011100 | 4,001**10 | 4,001*1*1
101 | 10 | 01 | 2,1011100 | 4,101**10 | 4,101*1*1
011 | 11 | 00 | 3,0111100 | 3,0111010 | 3,0111001
111 | 00 | 11 | 3,1110100 | 3,1110010 | 3,1110111

Finally we have reached t = τ. We can now decode our first message digit using step 4 of Algorithm 8.4.1. In this case, S(7) = {000} since d(000; 7) = 2 < d(s; 7) for all states s ≠ 000. Therefore the first message digit we decode is the rightmost digit in W(000; 7), namely 0. The rightmost digit in W(s; 7) is no longer used, so is discarded at t = 8 when constructing W(s; 8) (see step 3 of Algorithm 8.4.1).

The following table continues the decoding for several more ticks.

State |  |  |  |  |
s | t = 8 | t = 9 | t = 10 | t = 11 | t = 12
000 | 2,0000000 | 2,0000000 | 2,0000000 | 2,0000000 | 2,0000000
100 | 4,100**** | 4,100***0 | 4,100**** | 4,100***0 | 4,1000000
010 | 5,010**** | 5,010**** | 5,010**** | 5,010**** | 5,0100***
110 | 5,110**** | 5,110**** | 5,110**** | 5,110**** | 5,1100***
001 | 4,001**** | 4,0011101 | 4,0011100 | 6,001**** | 6,001****
101 | 4,101**** | 4,1011101 | 4,1011100 | 6,101**** | 6,101****
011 | 3,0111011 | 3,0111001 | 5,0111*** | 5,01110** | 5,01110**
111 | 3,1110011 | 5,111**** | 5,1110*** | 5,1110*** | 5,1110***
Decode to: | 0 | 0 | 0 | 0 | 0
< previous page | page_198 | next page >


page_199

< previous page | page_199 | next page >

Page 199

Example 8.4.3 Again consider the code C 1 of Example 8.4.2 and let w = 11 00 00 00 10 00... ↔ 1+x + x 8 be the received word. Again we will apply Algorithm 8.4.1 using a window size of τ(2) = 7 (see Example 8.3.5). The calculations are the same as in Example 8.4.2 until t = 5, at which point the x 8 term in w(x) comes in to play.

State | Output |  |  |
s | X 3 = 0 | X 3 = 1 | t = 4 | t = 5 | t = 6 | t = 7
000 | 00 | 11 | 2,0000000 | 3,0000000 | 3,000***0 | 3,0000***
100 | 11 | 00 | 2,1001000 | 3,1000000 | 1,1001110 | 3,1001001
010 | 10 | 01 | 3,0101000 | 2,0100100 | 4,010***0 | 2,0100111
110 | 01 | 10 | 3,1101000 | 4,110*100 | 4,110***0 | 2,1100111
001 | 01 | 10 | 4,0011000 | 1,0011100 | 3,0010010 | 5,001****
101 | 10 | 01 | 4,1011000 | 3,101*100 | 3,1010010 | 5,101****
011 | 11 | 00 | 1,0111000 | 4,011*100 | 4,0111*10 | 4,01110*1
111 | 00 | 11 | 3,1111000 | 4,111*100 | 4,1110*10 | 4,1110***
Decode to: |  |  |  |  |  | 1

In this case, at t = 7 the message digit 1 is decoded. If we assume the zero word was sent, then the third error introduced in w(x) has caused the decoder to decode incorrectly.

Exercises

8.4.4 Continue to decode w(x) in Example 8.4.3 for t = 8, 9, 10, 11 and 12. Will the decoded message digit be 0 for t ≥ 12?

8.4.5 Again using the convolutional code C 1 with g 1(x) = 1+x + x 3 and g 2(x) = 1+x 2+x 3, use Algorithm 8.4.1 with a window size of τ(2) = 7 to decode the following received words. Continue decoding until t = 9.

(a) w(x) = 1+x 3 ↔ 10 01 00 00...

(b) w(x) = 1+x + x 2 ↔ 11 10 00 00...

(c) w(x) = x 3+x 8+x 12 ↔ 00 01 00 00 10 00 10 00...

8.4.6 Algorithm 8.4.1 can be generalized to decode (n, k, m) convolutional codes as follows. (The state diagrams for such codes are defined in Exercise 8.2.14.)

1. Same as Algorithm 8.4.1.

2. For all t > 0 and for each state s 0, s 1, ..., sm−k define

where u ranges over all binary words of length k and where du is the distance between wt−1 and the output on the directed edge from state sk, ..., sm−k, u to state s in the state diagram.
< previous page | page_199 | next page >


page_2

< previous page | page_2 | next page >

Page 2

focus our attention on the construction of the encoder and the decoder. Our desire is to construct these in such a way as to effect:

1. fast encoding of information,

2. easy transmission of encoded messages,

3. fast decoding of received messages,

4. correction of errors introduced in the channel, and

5. maximum transfer of information per unit time.

The primary goal is the fourth of these. The problem is that it is not generally compatible with the fifth, and also may not be especially compatible with the other three. So any solution is necessarily a trade-off among the five objectives.

In our everyday communications among one another we standardly use words, spoken or written, made from a limited alphabet. We have information to communicate; we encode it into strings of words which we then speak or write. These are then sent across a channel, the channel normally being the space from mouth to ear or from pen to paper to eye. The noise might be caused by poor speech, bad hearing, incorrect grammar, a loud stereo, competing speech, misspelling, misreading, or a faulty typewriter. The decoder is our reading (or hearing) and understanding of the received messages.

We have built-in error-correcting devices that that we don't even think about. Suppose we receive the message ''Apt natural. I have a gub." which is a hold-up note in Woody Allen's "Take the Money and Run." Since our language does not use all possible words of any given length, we hopefully recognize that "gub" is not a word. We may safely assume that the transmitted word was close to "gub" in some sense. So it was more likely to have been "gut" or "gun" or "tub" than say "firetruck" or "rat." It is only the context of the message though that lets us choose "gun" as the most likely word. "Apt" is a perfectly good word, but again from the context we are led to correct it to "act." And if we happen to be literate, we will also correct "natural" to "naturally," even though this was probably an error attributed to the source and not to the noise on the channel.

Of these types of errors, we can probably only deal with the first: that is choosing the most likely word transmitted. The standard method for combating errors is through redundancy. Many businesses these days commonly add check digits to identification numbers; these are extra digits that are used to check the correctness of data or of account numbers. This is probably the most commonly recognized method of coding in real life. We shall deal with more sophisticated but similar ideas.

1.2 Basic assumptions

We state some fundamental definitions and assumptions which will apply throughout the text.
< previous page | page_2 | next page >


page_20

< previous page | page_20 | next page >

Page 20

Example 1.11.6 Consider the code C = {000, 111} with IMLD Table 1.1. All possible error patterns u are in the first column. For a given u, all sums v + u as v ranges over C are in the second and third columns of the row labeled by u. If none of these entries are in C (that is, neither is 000 or 111), then C detects u. Thus C detects the error patterns 100, 010, 001, 110, 101, and 011, as can be seen by inspecting rows 2 through 7 of the table, but not the error patterns 000 or 111.

Exercises

1.11.7 Determine the error patterns detected by each code in Exercise 1.9.7 by using the IMLD tables constructed there.

An alternative and much faster method for finding the error patterns that code C can detect is to first find all error patterns that C does not detect; then all remaining error patterns can be detected by C. Clearly, for any pair of codewords v and w, if e = v + w then e cannot be detected, since v + e = w, which is a codeword. So the set of all error patterns that cannot be detected by C is the set of all words that can be written as the sum of 2 codewords.

Example 1.11.8 Consider the code {000, 111}. Since

000 + 000 = 000, 000 + 111 = 111 and 111 + 111 = 000,

the set of error patterns that cannot be detected is {000, 111}. Therefore all error patterns in K 3\{000, 111} can be detected.

Example 1.11.9 Let C = {1000, 0100, 1111}. Since 1000 + 1000 = 0000, 1000 + 0100 = 1100, 1000 + 1111 = 0111 and 0100 + 1111 = 1011, the set of error patterns that cannot be detected by C is {0000, 1100, 0111, 1011}. Therefore all error patterns in K 4\{0000, 1100, 0111, 1011} can be detected.

Exercises

1.11.10 Find the error patterns detected by each of the following codes and compare your answers with those Exercises 1.11.7.

(a) C = {101, 111, 011}

(b) C = {000, 001, 010, 011}

(c) C = {0000, 0001, 1110}

(d) C = {0000, 1001, 0110, 1111}

(e) C = {00000, 11111}

(f) C = {00000, 11100, 00111, 11011}

(g) C = {00000, 11110, 01111, 10001}

(h) C = {000000, 101010, 010101, 111111}

There is also a way of determining some error patterns that code C will detect without any manual checking. First we have to introduce another number associated with C.

For a code C containing at least two words the distance of the code C is the smallest of the numbers d(v, w) as v and w range over all pairs of different

< previous page | page_20 | next page >


page_200

< previous page | page_200 | next page >

Page 200

3. (a) If d(sk,...,sm−k,u;t − 1)+du < d(sk,...,sm−k, v;t − 1)+dv for all v ≠ u then form W(s;t) from W(sk,...,sm−k, u;t − 1) by deleting the rightmost k digits from it and adding the leftmost k digits of s to it.

(b) If d(sk,...,sm−k, u;t − 1) is not the smallest value for a unique choice of u, then we could form W(s;t) by choosing any such u and proceeding as in 3(a). Alternatively, as in Algorithm 8.4.1, we can take a combination of all the walks W(sk,...,sm−k, u;t − 1) for which d(sk,...,sm−k, u;t − 1) is a minimum, placing an * in any position where 2 such walks disagree.

4. For t ≥ τ, let S(t) = {s | d(s;t) ≤ d(s′;t) for all states s′}. Decode the message digits m1,t, m2,t...mk,t where mi,t is the ith digit in the rightmost k digits of W(s,t), for all s ∈ S(t), unless two such walks disagree in the ith position in which case mi,t = *.

Check that this is a generalization of Algorithm 8.4.1.

There are several comments that should be made concerning Algorithm 8.4.1. First, there are other ways to define the decoding step, step 4, of Algorithm 8.4.1. For example, it could be argued that decoding should not take place until the walks to each state agree on the rightmost digit (that is, the digit used for decoding). However in such an algorithm we might need to wait for many ticks before any decoding can be done, thus raising the problem of open ended storage requirements. Another variation of step 4 of Algorithm 8.4.1 would be to delete each walk in which the rightmost digit disagrees with the message digit currently being decoded (because such walks choose to move to a different state). This decoding technique poses theoretical problems in the analysis of the algorithm, as it is conceivable that such a decoding algorithm might itself impose infinitely many decoding errors after a finite burst of errors during transmission.

Second, we should ask if we can still prove a result as strong as Theorem 8.3.4 for the truncated Viterbi decoder of Algorithm 8.4.1. The answer is no, because this truncated Viterbi decoding algorithm takes some time to recover from errors imposed on the codeword during transmission. To see why there should be a difference between the two algorithms, consider tick t = 2 in Example 8.4.2. When using the truncated Viterbi decoding algorithm, the walk staying in state 000 ''remembers" the 2 errors that occurred in w at tick t = 1 when wt−1 = 11. It remembers the errors because d(000; 2) = 2. In Example 8.4.7 we will see that the effect of these 2 errors lasts until t = 12. On the other hand, for the exhaustive decoding algorithm the 2 errors affected the decoding decision at tick t = 1, but had no effect for t ≥ 2 (recall that for t ≥ 2, the walks of length τ from state 000 were all compared to w t−1,w t,...,w t+τ−2 = 00...0, a portion of the received word that agrees exactly with the walk staying in the zero state).

We shall now be more precise about how long errors during transmission will affect the decoding when using the truncated Viterbi decoder with window size τ(e) defined by Algorithm 8.4.1. We begin with some definitions. Let w(s,s′) be

< previous page | page_200 | next page >


page_201

< previous page | page_201 | next page >

Page 201

the weight of a least weight path from s to s′ in the state diagram. Suppose that state s(t) is the correct state at some tick t (that is, s(t) is the state the codeword sent is in at tick t). Then the decoder is defined to be e-ready at tick t if the following conditions hold:

(1) d(s′;t) ≥ d(s(t);t)+min{1+e,w(s(t),s′)} for all states s′ ≠ s(t), and

(2) if w(s(t),s′) < 1+e then W(s′;t) = s′v (of length τ), where v is defined by W(s(t);t) = s(t)v.

Example 8.4.7 Consider Example 8.4.2. The correct state for all t ≥ 1 is s(t) = 000, since we are assuming that the codeword sent was the zero word. Since in this case m = 3 is small, it is not hard to calculate w(s(t),s′) = w(000,s′) for all states s′ ≠ 000 from the state diagram:

w(000,100) = 2, w(000,010) = 3, w(000,001) = 4, w(000,110) = 3,
w(000,101) = 4, w(000,011) = 3, w(000,111) = 3.

The first time that the decoder is 2-ready in Example 8.4.2 is at t = 12. To see this, notice the following observations.

At t = 10, d (001; 10) = 4 < 5 = d(000; 10)+min{1+e,w(000, 001)}, so (1) in the definition of e-ready is not satisfied.

At t = 11, all states satisfy (1) in the definition of e-ready, but w(000, 100) = 2 < 3 = 1+e and W(100; 11) = 100**** ≠ 100v (since W(000; 11) = 0000000= s(11)v, so v = 0000).

At t = 12, all states satisfy (1). s′ = 100 is the only state with w(000,s′) < 1+e, and W (100; 12) = 1000000 = s′0000 = s′ v.

The following result demonstrates the value of being e-ready.

Theorem 8.4.8 Let C be a non-catastrophic convolutional code which is decoded using the truncated Viterbi decoder of Algorithm 8.4.1. At tick t, if the decoder is e-ready, then correct decoding will occur if at most e errors are subsequently made during transmission.

This result makes sense of the name e-ready. However, clearly this result is still much weaker than Theorem 8.3.4. A guard space is defined to be a time period of error-free transmission following a burst of errors. To obtain a result comparable to Theorem 8.3.4 would require knowing how long a guard space is required before the decoder is e-ready. For the exhaustive decoder, Theorem 8.3.4 says that the guard space required is 0 (if we think of e-ready as meaning that any subsequent error pattern of weight at most e will still result in a received word that is decoded correctly). It turns out that it can be proved that the guard space required for our truncated Viterbi decoder to become e-ready after a burst of errors is finite, and that the length of the guard space is known for some convolutional
< previous page | page_201 | next page >


page_202

< previous page | page_202 | next page >

Page 202

codes where m is small. The closest we can get to Theorem 8.3.4 is the following result.

Theorem 8.4.9 Let C be a non-catastrophic convolutional code which is decoded using the truncated Viterbi decoder with window size τ(e) of Algorithm 8.4.1. If the error pattern can be partitioned into bursts of errors, each of weight at most e and each followed by a sufficiently long (finite) guard space, then the decoder will decode correctly.

Exercises

8.4.10 (a) Apply Algorithm 8.4.1 using a window size of τ(2) = 6 to decode the received word w = 11 00 00... ↔ 1+x = w(x) that was originally encoded using the (2,1,2) convolutional code with generators g 1(x) = 1+x 2 and g 2(x) = 1+x + x 2. Continue decoding to show that the decoder is 2-ready at t = 10, assuming that the zero word is the codeword that was sent (so the correct state s(t) is the zero state, for all t).

(b) At t = 9 the decoder is not 2-ready, so Theorem 8.4.8 does not guarantee that any subsequent error pattern of weight at most e = 2 will be decoded correctly. Show that if at t = 10 and at t = 11, the digits in the received word are each changed to 10, (so the received word is w(x) = 1+x + x 18+x 20) then the decoder decodes an * at t = 12.

8.4.11 For each of the received words in Exercise 8.4.5, find the smallest t such that at tick t the decoder is 2-ready.

Finally, we return to the calculation of d(C) and τ(e). In both cases we have to find weights of walks from the zero state that immediately leave the zero state. To find d(C) we want the weight of such a walk that has the smallest possible weight, to find τ(e) we want the length x such that all such walks of length at least x have weight more than 2e. We can modify the truncated Viterbi decoder of Algorithm 8.4.1 to do both these tasks. First, by assuming that the zero word is sent, the distance function is simply measuring the weights of the walks, as we now require. Second, to force the walks to immediately leave the zero state, we simply define d(00...0; 1) = ∞. This has the effect of removing the walk that stays at the zero state from consideration, since the only walk remaining with d(s; 1) being finite is the walk to s = 100...0 (that is, the walk that immediately leaves the zero state). Third, we need not store the walks W(s;t) since they are of no concern in these calculations. Fourth, we need to recognize the answer! For any non-catastrophic code, every finite weight walk returns to the zero state and remains there. At each tick t,d(s; t) is the weight of a least weight walk of length t that immediately leaves the zero state (in view of the second consideration) and ends in state s. Also, if at tick t d(s; t) ≥ d(00...0; t) for all states s then d(00...0;t′) = d(00...0;t) for all t′ ≥ t (because from Step 2 of Algorithm 8.4.1 it is clear that for any state

< previous page | page_202 | next page >


page_203

< previous page | page_203 | next page >

Page 203

s′, d(s′;t′) ≥ mins{d(s;t′ − 1)}). Therefore d(C) = d(00...0; t). Similarly, once d(s;t) > 2e for all states s, all walks of length t that immediately leave the zero state have weight more than 2e. So τ(e) is the first tick t such that d(s;t) > 2e for all states s. Therefore we have the following modification of Algorithm 8.4.1 to find d(C) and τ(e).

Algorithm 8.4.12 (finding d(C) and τ(e) for a non-catastrophic convolutional code) Let wt(s;s′) be the weight on the edge in the state diagram directed from s to s′.

1. If t = 1 then define

2. For t > 1 and for each state s = s0,...,sm−1, define

3. If d(00...0;t) ≥ d(s;t) for all states s then d(C) = d(00...0; t).

4. If d(s;t) > 2e for all states s and if d(s′;t − 1) ≤ 2e for some state s′ then τ(e) = t.

Remark If we assume that w = 000... is the received word, then Algorithm 8.4.12 is essentially Algorithm 8.4.1 except that d(00...0;1) is defined to be ∞ and W(s;t) is never calculated.

Example 8.4.13 We find d(C) and τ(e), 1 ≤ e ≤ [image] (d(C) − 1)/2 [image] for the convolutional code C 1 with generators g 1(x) = 1+x + x 3 and g 2(x) = 1+x 2+x 3. (These were previously calculated in Example 8.3.6 and in the text before Exercise 8.3.5.) We follow the format used in Example 8.4.2 when applying Algorithm 8.4.1.
< previous page | page_203 | next page >


page_204

< previous page | page_204 | next page >

Page 204

At t = 10, d(000; 10) ≥ d(s; 10) for all states s, so d(C) = d(000; 10) = 6 (by Step 3 of Algorithm 8.4.11).

Since 1 ≤ e ≤ [image] (d(C) − 1)/2 [image], we consider e = 1 and e = 2 in turn.

If e = 1 then d(s; 2) > 2e for all states s and d(100; 1) = 2 ≤ 2e. Therefore τ(1) = 2 (by Step 4 of Algorithm 8.4.12).

If e = 2, then d(s; 7) > 2e for all states s and d(100; 6) = 4 ≤ 2e. Therefore τ(2) = 7 (by Step 4 of Algorithm 8.4.12).

Exercises

8.4.14 For each of the convolutional codes C with the following generators, use Algorithm 8.4.12 to find d(C) and τ(e) for 1 ≤ e ≤ [image] (d(C) − 1)/2 [image]. Compare your answers to those of Exercises 8.3.3 and 8.3.6.

(a) g1(x) = 1+x 2 and g2(x) = 1+x + x 2

(b) g1(x) = 1+x + x 2+x 3 and g2(x) = 1+x 2+x 3

(c) g1(x) = 1+x 3+x 4 and g2(x) = 1+x + x 2+x 4.
< previous page | page_204 | next page >


page_205

< previous page | page_205 | next page >

Page 205

Chapter 9
Reed-Muller and Preparata Codes

9.1 Reed-Muller codes

In Chapter 3, we gave a method of constructing Reed-Muller Codes, RM(r,m) and established many basic properties. Recall that these are linear (n,k,d) codes with n = 2m, k = [image], and d = 2m−r. In this section we will give an alternate construction of these codes; one that is better suited to decoding.

As with Reed-Solomon and other codes we will label the coordinate positions of words of length n = 2m, this time by vectors in Km. As a matter of convenience and consistency we will label coordinate position i with vector ui ∈ Km, where ui is the binary representation of the integer i, with digits in reverse order (low order digit first); call this the standard ordering of Km.

Example 9.1.1 Standard ordering for

K 2 is (00, 10, 01, 11), and for K 3 is (000, 100, 010, 110, 001, 101, 011, 111).

Any function, f, from Km to {0, 1} has a unique representation or vector form [image] [image] is the standard ordering of vectors in Km as described above.

We are interested in a certain class of basic functions. Given a subset I [image] {0, 1,...,m − 1}, define a function

(fI is a function mapping Km to {0, 1}.) Define vl to be the corresponding vector form of fI.

Example 9.1.2 Let m = 3, so n = 23.

(a) If I = {1, 2} then fl(x 0,x 1,x 2) = (x 1 + 1)(x 2 + 1). The vector form of f{1,2}(x 0, x 1, x 2) is formed by taking each of the elements x 0 x 1 x 2 ∈ K 3 (using the standard ordering) and evaluating f{1,2}(x 0, x 1, x 2). So f{1,2}(0,0,0)

< previous page | page_205 | next page >


page_206

< previous page | page_206 | next page >

Page 206

= 1, f{1,2}(1,0,0) = 1, f{1,2}(0, 1, 0) = 0, f{1,2}(1, 1, 0) = 0, f{1,2}(0, 0, 1) = 0, f{1,2}(1, 0, 1) = 0, f{1,2}(0, 1, 1) = 0 and f{1,2}(1, 1, 1) = 0. Therefore vI = 11000000.

(b) If I = {0} then fI(x 0, x 1, x 2) = (x O + 1) and vI = 10101010.

(c) If I = Ø, then f Ø(x 0, x 1, x 2) = 1 and vI = 11111111.

There are two facts about the function fI, which we will use later. First of all, fI(x 0, x 1,...,x m−1) = 1 if and only if xi = 0 for all i ∈ I. Thus in Example 9.1.2(a), I = {1, 2}, fI(x 0, x 1, x 2) = (x 1+1)(x 2+1) and f(x 0, 0, 0) = (0+1)(0+ 1) = 1 for x 0 ∈ {0, 1}. Second, for each ui ∈ Km fI(ui)fJ(ui) = fI∪J(ui) and thus

We will use Zm to denote the set of integers {0, 1, 2,...,m − 1}.

Exercises

9.1.3 Let m = 4, so n = 24. For each of the following choices of I, subsets of Z 4, find fI and vI:

(a) I = {0, 3}

(b) I = {0, 1, 3}

(c) I = {1}

(d) I = {2, 3}

(e) I = Ø

(f) I = Z 4

9.1.4 Let m = 5, so n = 25. For each of the following choices of I, subsets of Z 5, find fI and vI:

(a) I = {0, 2, 4}

(b) I = {0, 1, 3, 4}

(c) I = {1}

(d) I = {1, 2, 4}

(e) I = Ø

(f) I = Z 5

9.1.5 Let I be a subset of Zm. Use the first fact above to show that wt(vI) = 2m - |I|.

9.1.6 If v is a linear combination of vectors of the form vI, when will v have even weight?

9.1.7 Let m = 4, so n = 24. For I = {0, 1, 3} and J = {2, 3}, compute vI · vJ.
< previous page | page_206 | next page >


page_207

< previous page | page_207 | next page >

Page 207

Figure 9.1: The generator matrix G4,4.

The Reed-Muller code RM(r,m) can be defined as the linear code 〈{vI | I [image] Zm,|I| ≤r}〉. We claim that S = {vI | I [image] Zm,|I| ≤ r} is a linearly independent set (see Exercise 9.1.10), and thus a basis for RM(r,m). By counting the number of words vI with I [image] Zm and |I| ≤ r, we have that for RM(r,m),

and clearly n = 2m. Of course the words vI can be arranged in any order to form a generating matrix for RM(r,m). We define the generating matrix Gr,m of RM(r,m) to be in canonical form if the rows are ordered so that vI comes before vJ if |I| < |J|, or if |I| = |J|, fI(uj) < fJ(uj) and fI(ui) = fJ(ui) for i > j.

Example 9.1.8 The generator matrix for R(4, 4) in canonical form is G 4, 4 given in Figure 9.1. For convenience, we have written v{3} as v 3 (and similarly for the other subscripts). This ordering follows from the definition as the following examples indicate. If I = {3} and J = {2, 3}, since |I| < |J|, v3 = v1 precedes v 2,3 = vJ. If I = {2, 3} and J = {0, 2} then fI(ui) = fJ(ui) for i > 10 but fI(u10) = 0 < 1 = fJ(u10) (of course, u 10 in the standard ordering of K 4 is 0101). Therefore v 2, 3 = vI precedes v 0,2 = vJ.

Now it is easy to see that G 0,4, G 1,4, G 2,4 and G 3,4 are simply the submatrices of G 4,4 formed by the first [image] and [image] rows respectively.
< previous page | page_207 | next page >


page_208

< previous page | page_208 | next page >

Page 208

Exercises

9.1.9 Find (a) G 2,3 (b) G 2,4 (c) G 3,5 (d) G 0,10

9.1.10 Show that for all r ≤ m, {vI ||I| ≤ r, I [image] Zm} is a linearly independent set. (Hint: Arrange the words in this set so that vI comes before vJ if, for some j, fI(ui) = fJ(ui) for j + 1 ≤ i ≤ m and fI(uj) > fJ(uj). Or, more formally, use induction on m and r.)

Encoding is done, as for any linear code, by multiplying a message by Gr,m. Then any codeword c can be written as

(where the message digits are labeled mI to correspond to the rows vI of Gr,m).

Example 9.1.11 Encoding the following messages m using G 2,4 results in the corresponding codeword c.

(a) If m = 1 0000 001000 (so m Ø = 1 and m 0,3 = 1) then

c = v Ø + v 0,3 = 0101010111111111.

(b) If m = 0 0101 001001 (so m2 = m 0 = m 0,3 = m 0,1 = 1) then

c = v 2 + v 0 + v 0,3 + v 0,1 = 0111100011010010.

Exercises

9.1.12 Encode the following messages using G 2,4.

(a) 0 0101 000000

(b) 0 0000 000001

(c) 0 0100 001000

9.2 Decoding Reed-Muller codes

We shall decode the Reed-Muller codes using an easily implementable process known as majority logic decoding. To understand this, we shall need some preliminary results. For any I [image] Zm, define Ic = Zm\I to be the complement of I in Zm.

Let HI = {u ∈ Km | fI(u) = 1}. Recall fI(x0,...,xm−1) = Πi∈I(xi+1) = 1 if and only if xi = 0 for all i ∈ I. Clearly if x, y ∈ HI, then xi = yi = 0 = xi + yi for all i ∈ I, thus x + y ∈ HI. Therefore HI is a subspace of Km.

For any u = (x0, x1,...,xm−1) ∈ Km and for any t = (t0, t1,...,tm−1) ∈ Km, define another function fI,t(x0, x1,...,xm−1) = fI(x0 + t0,...,xm−1 + tm−1) = fI(x + t) and define vI,t to be the vector form of fI,t.
< previous page | page_208 | next page >


page_209

< previous page | page_209 | next page >

Page 209

We will be interested in finding [image], and so we need to count the number of words u ∈ Km for which [image]. By the definition of HI, fI,t(u) = fI(u+t) = 1 if and only if u + t = u′ ∈ HI, or equivalently u = u′ + t ∈ HI + t, where HI + t is the coset of HI determined by t. And the value of [image] remains the same for all choices of xk ∈ {0,1}, k ∈ Zm\(I ∪ Jc). As there are [image] such choices for u as u ranges over the elements of Km, the number of times

is a multiple of [image] and thus even unless | I∪Jc| = m; that is, unless I ∪ Jc = Zm. However, if we assume that |I| ≤ |J| then |Jc| ≤ |Ic|. So |I ∪ Jc | = |I| + |Jc| – |I ∩ Jc| < m unless I = J. If I = J then there is only one u ∈ Km for which [image], namely the u for which xi = si for all i ∈ I and xi = ti for all i ∈ Ic.

Of course, finding the number of places where [image] immediately gives [image], so we have the following result.

Lemma 9.2.1 Let I and J be subsets of Zm, with |I| ≤ |J|. For any [image] and for any t ∈ HJ,

= 1 if and only if I = J.

Now we can easily obtain the following result which is the basis of the decoding scheme that we shall use.

Corollary 9.2.2 If c is a codeword in RM(r,m) and if |J| = r then mJ = [image] for any t ∈ HJ.

Proof: If |J| = r then for any t ∈ HJ,

since by Lemma 9.2.1, the only dot product in the sum which is not zero is the one with I = J.

Lemma 9.2.3 Let J [image] Zm. For any word e (of length 2m), [image] for at most wt(e) values of t ∈ HJ.

Proof: Recall that for any subspace S of Km, two words are in the same coset in the coset decomposition of S precisely when their sum is a word in S. Also, HJ is a subspace of Km and the only word in both HJ and [image] is the zero word. It follows from these observations that no two words of HJ occur together in a coset of [image]. Therefore as t ranges over the elements of HJ, [image] + t forms the coset decomposition of [image].
< previous page | page_209 | next page >


page_21

< previous page | page_21 | next page >

Page 21

codewords in C. Note that since d(v, w) = wt(v+w), the distance of the code is the smallest value of wt(v+w) as v and w, v ≠ w range over all possible codewords.

The distance of a code has many of the properties of Euclidean distance; this correspondence may be useful to assist in understanding the concept of the distance of a code.

Example 1.11.11 Let C = {0000, 1010, 0111}. Then d(0000, 1010) = 2,d(0000, 0111) = 3, and d(1010, 0111) = 3. Thus the distance of C is 2.

Exercises

1.11.12 Find the distance of each of the following codes.

(a) C = {101, 111, 011}

(b) C = {000, 001, 010, 011}

(c) C = {0000, 0001, 1110}

(d) C = {0000, 1001, 0110, 1111}

(e) C = {00000, 11111}

(f) C = {00000, 11100, 00111, 11011}

(g) C = {00000, 11110, 01111, 10001}

(h) C = {000000, 101010, 010101, 111111}

1.11.13 Find the distance of the code formed by adding a parity check digit to Kn.

Now we can state a theorem which helps to identify many of the error patterns a code will detect.

Theorem 1.11.14 A code C of distance d will at least detect all non-zero error patterns of weight less than or equal to d − 1. Moreover, there is at least one error pattern of weight d which C will not detect.

Notice that C may detect some error patterns of weight d or more, but does not detect all error patterns of weight d.

Proof: Let u be a nonzero error pattern with wt(u) ≤ d − 1, and let v be in C. Then

d(v,v+u) = wt(v+v+u) = wt(u) < d.

Since C has distance d, v+u is not in C. Therefore C detects u. From the definition of d, there are codewords v and w in C with d(v, w) = d. Consider the error pattern u = v+w. Now w = v+u is in C, so C will not detect the error pattern u of weight d.

A code is an t error-detecting code if it detects all error patterns of weight at most t and does not detect at least one error pattern of weight t+1. So, in view of Theorem 1.11.14, if a code has distance d then it is a d − 1 error-detecting code.

Example 1.11.15 The code C = {000, 111} has distance d = 3. By Theorem 1.11.14, C detects all error patterns of weight 1 or 2, and C does not detect the

< previous page | page_21 | next page >


page_210

< previous page | page_210 | next page >

Page 210

The result now follows since if t 1 ≠ t 2 are two different elements of [image], we have just shown that ( [image] +t 1)∩( [image] +t 2) = Ø, so [image] and [image] have no positions in common where both the digits are 1. Therefore each of the wt(e) non-zero digits in e affects at most one of the values of [image] as t ranges over the elements of HJ.

We can now obtain a decoding algorithm as follows. Let w = c+e be a received word where c is a codeword in RM(r,m); so [image], where |I| ≤ r. Let J [image] Zm be a set of size r. Then by Lemma 9.2.3, [image] for at least |HJ| − wt(e) values of t in HJ; for such values of t we have that

So if 2 wt(e) < |HJ|, as t ranges over the elements of HJ, more than half of the [image] will be mJ.

Once mJ has been calculated in this way for all J [image] Im with |J| = r, let w(r − 1) = [image]. Now w(r − 1) can be decoded by treating it as a received word that was encoded using RM(r − 1,m). This process can be continued until mJ has been found for all J [image] Im with |J| ≤ r.

Before summarizing this algorithm we make note of the fact that this algorithm corrects all error patterns of weight less than |HJ|/2 where |J| ≤ r. However by Exercise 9.1.5, |HJ| = wt(vJ) = 2m−|J|. So all error patterns of weight less than 2m−r−1 are corrected and therefore RM(r,m) has minimum distance at least 2m−r. However, if I [image] Zm and |I| = r then v 1 is a codeword in RM(r,m) and has weight 2m−r, so we have another proof of the following result:

Lemma 9.2.4 The minimum distance of RM(r,m) is 2m−r.

Algorithm 9.2.5 (majority logic decoding of RM(r,m)) To decode a received word, proceed as follows.

1. Let i = r and let w(r) = w.

2. For each J [image] Zm with |J| = i, calculate [image] for each t ∈ HJ until either 0 or 1 occurs more than 2m−i−1 times and let mJ be 0 or 1 respectively; if both 0 and 1 occur more than e = 2m−r−1 − 1 times then ask for retransmission.

3. If i > 0 then let [image] where |J| = i. If w(i − 1) has weight at most e = 2m−r−1 − 1 then set mJ = 0 for all J [image] Zm with |J| ≤ r and stop. Otherwise replace i with i − 1 and return to step 2. (If i = 0 then mJ has been calculated for all J [image] Zm with |J| ≤ r, so the most likely message has been found.)
< previous page | page_210 | next page >


page_211

< previous page | page_211 | next page >

Page 211

Figure 9.2: Majority logic decoding of RM(2, 4), Step 1 (see Example 9.2.6).

Example 9.2.6 Use Algorithm 9.2.5 to decode the received word w = 0101011110100000 that was originally encoded using G 2,4.

Begin with i = r = 2 and w(2) = w. From the computations of Figure 9.2 we see that m 2,3 = 0,m 1,3 = 0,m 0,3 = 0,m 1,2 = 0,m 0,2 = 1, and m 0,1 = 0. Then

w(1) = w(2) + v 0,2 = 11110111 0000 0000

and i = 1.

Again from the computations of Figure 9.3 we conclude that m 3 = 1, m 2 = 0, m 1 = 0, and m 0 = 0. Let w(0) = w(1) − v 3 = 0000 1000 0000 0000 and let i = 0. Since w(0) has weight at most e = 1, set m Ø = 0 and stop.

So the most likely message is 0 1000 000010 (since messages were encoded using G 2,4).

Exercises

9.2.7 Messages are encoded using the generator matrix G 2,4. If possible decode the following received words:
< previous page | page_211 | next page >


page_212

< previous page | page_212 | next page >
Page 212 Figure 9.3: Majority logic decoding of RM(2,4), Step 2 (see Example 9.2.6).
< previous page | page_212 | next page >


page_213

< previous page | page_213 | next page >

Page 213

(a) w = 0111 0101 1000 1000

(b) w = 0110 0110 0001 0000

(c) w = 0101 1010 0100 0101

(d) w = 1110 1000 1001 0001

(e) w = 0011 0000 0011 0100

(f) w = 1001 0110 0101 1010

(g) w = 1010 1000 1010 0000

(h) w = 0011 1100 0001 1100

(i) w = 1001 1101 0001 1101

9.2.8 Messages are encoded using the generator matrix G 2,5. If possible decode the following received words:

(a) w = 1100 1000 1110 0000 1100 0000 1100 0100

(b) w = 0101 0111 0101 1000 1000 1000 0111 1010

(c) w = 0011 0011 1111 0011 0011 0011 1111 1111

(d) w = 0100 0000 1111 1111 0000 1100 0000 1111

(e) w = 1001 0101 0110 1001 1001 0111 0110 1010

(f) w = 0011 1111 0011 0011 1100 1100 1100 0100

(g) w = 0100 0100 1111 1111 0000 1100 0000 1111

9.3 Extended Preparata codes

In this section we shall refer to the coordinate positions of words of length 2r by using the elements of GF(2r). This labeling of the positions was also used when considering B C H codes, although there the field element 0 was never used as a label. So for any subset U consisting of elements of GF(2r) let X(U) be the word of length 2r which is

(where, as usual β is a primitive element of GF(2r)).

Example 9.3.1 Let β be a primitive element of GF(23). Then

For any element α and any subset U of elements of GF(2r), let U + α = {u + α | u ∈ U} and let α U = {αu | u ∈ U}. Also, for any two subsets U and V of elements of GF(2r), define the symmetric difference U Δ V of U and V to be {x | x ∈ U or x ∈ V but x ∉ U ∩ V}. Then it is easy to see that χ (U) + χ (V) = χ (U Δ V).
< previous page | page_213 | next page >


page_214

< previous page | page_214 | next page >

Page 214

Example 9.3.2 Let GF(23) be constructed using 1+x + x 3. Let U = {β 2,β 5,β 6} and let V = {β 5, 0}. Then

Definition 9.3.3 The extended Preparata code P(r) is the set of codewords of the form χ (U) followed by χ (V) where U and V are subsets of elements of G F(2r) which satisfy

(i) |U| and |V| are even,

(ii)

(iii) [image], and

(iv) r is odd.

We denote such codewords by [χ (U), χ(V)]. Since both χ (U) and χ (V) have length 2r, P(r) is a code of length 2r+1.

Example 9.3.4 Construct GF(23) using 1+x + x 3. Let U = {β, β 2, β 5, 0} and let V = {β 0, β, β 2, β 3, β 6, 0}. Clearly (i) and (iv) of Definition 9.3.3 are satisfied. Also

so (ii) is satisfied, and

so (iii) is satisfied since β 2 + (β 0)3 = β 6. Therefore [χ (U), χ (V)] = 01100101 11110011 is a codeword in P(3).
< previous page | page_214 | next page >


page_215

< previous page | page_215 | next page >

Page 215

Notice that whether or not 0 is an element of U or of V does not affect any of the calculations in (ii), (iii) or (iv) of Definition 9.3.3. So 0 is only used in U or V to make |U| or |V| even. Therefore the digit in position 2r−1 of χ (U) is simply a parity check digit for χ(U), and similarly the digit in position 2r−1 of χ (V) is a parity check digit for χ (V).

It turns out that P(r) is not a linear code as the following result suggests (see Theorem 9.3.18). Therefore P(r) does not have a dimension.

Lemma 9.3.5 Suppose that [χ(U),χ(V)] and [χ(A),χ(B)] are codewords in P(r). Let α = Σu∈U u. Then [χ(UΔA+α),χ(VΔB)] is also a codeword in P(r).

Proof: We check that conditions (i), (ii), and (iii) of Definition 9.3.3 are satisfied by [χ (UΔA+α), χ(VΔB)].

(i) Since |U|, |V|, |A|, and |B| are even,

(ii) First notice that for any subsets I and J of elements of GF(2r),Σx∈IΔJ x =Σx∈I x+Σx∈J x since any element βi in both I and J is counted twice on the right hand side and not at all on the left, but 2βi = 0. Therefore

(iii)
< previous page | page_215 | next page >


page_216

< previous page | page_216 | next page >

Page 216

But α = Σy∈U y and so Σy∈V y=Σy∈U y = α Also, (Σy∈V y)2 = Σy∈V y 2 and again we use the fact that |U| and |A| are even. Therefore the above expression reduces to simply

Even though P(r) is a non-linear code, it does have some properties in common with linear codes.

Definition 9.3.6 A code is distance invariant if for any pair of codewords c 1 and c 2, the number of codewords distance i from c 1 equals the number of codewords distance i from c 2 for 1 ≤i≤n.

So for any distance invariant code that contains the zero word, it follows immediately that the minimum distance is the weight of a non-zero codeword of smallest weight.

Corollary 9.3.7 P(r) is distance invariant.

Proof: Let [χ(U),χ(V)] and [χ(A),χ(B)] be codewords in P(r) that are distance i apart. By Lemma 9.3.5, [χ(UΔU+α),χ(VΔV)] and [χ(UΔA+α), χ(VΔB)] are both codewords, and it's not hard to see they must also be distance i apart. Since UΔU=∅,[χ(UΔU+α),χ(VΔV)] is the zero word and so [χ(UΔA+α),χ(VΔB)] is a codeword of weight i.

The following lemma lists various properties of P(r). These can be proved using the same ideas as were used in proving Lemma 9.3.5, and so are left as exercises.

Lemma 9.3.8 Suppose that [χ(U),χ(V)] is a codeword in P(r). Then P(r) also contains the following codewords:

(i) [χ(V),χ(U)],

(ii) [χ(U+α),χ(V+α)] for any α ∈ GF(2r) and

(iii) [χ(αU),χ(αV)] for any α ∈ GF(2r, α ≠ 0.
< previous page | page_216 | next page >


page_217

< previous page | page_217 | next page >

Page 217

Example 9.3.9 From Example 9.3.4, [χ(U),χ(V)] is a codeword in P(3) where U = {β,β 2,β 5,0} and V = {β 0,β,β 2,β 3,β 6,0}. By applying Lemma 9.3.8 with α = β 3, we find that the following are also codewords:

(i) [χ(V),χ(U)] = 11110011 01100101,

Exercises

9.3.10 Apply Lemma 9.3.8 to the codeword [χ(U),χ(V)] defined in Example 9.3.9 by using

(a) α = β 0

(b) α = β

(c) α = β 6

9.3.11 Why is [χ(αU),χ(αV)] not a codeword when α = 0 (this possibility is excluded in Lemma 9.3.8)?

9.3.12 Show that the three words formed in Example 9.3.9 satisfy Definition 9.3.6.

We can use Lemma 9.3.8 to simplify the problem of finding the minimum distance of P(r), but first we need one more lemma which indicates the reason that we require r to be odd.

Lemma 9.3.13 If β is a primitive element of GF(2r) then β 3 is a primitive element if r is odd and is not primitive if r is even.

Proof: We know that β i is primitive if and only if the greatest common divisor of i and 2r − 1 is 1; that, is i and 2r − 1 are relatively prime (Exercise 5.1.18). If r is odd then 2r − 1 ≡ 1 (mod 3) and if r is even then 2r − 1 ≡ 0 (mod 3) (this is easy to prove, say by induction). So if r is even then we can write 2r − 1 = 3x for some integer x and β 3 is not primitive. However if r is odd then write 2r − 1 = 3x+1 and clearly β 3 is a primitive element.

Corollary 9.3.14 If r is odd then for each nonzero element x of GF(2r) there is a unique element y (called the cube root of x) such that y 3 = x.

Theorem 9.3.15 P(r) has minimum distance 6.
< previous page | page_217 | next page >


page_218

< previous page | page_218 | next page >

Page 218

Proof: Since P(r) is distance invariant, P(r) contains a codeword of weight d, say [χ(U),χ(V)]. Then

d = wt(χ(U)) + wt(χ(V)) = |U| + |V|.

By (i) of Definition 9.3.3, d is even so we only need to show that d ≠ 2, d ≠ 4 and that there is a codeword of weight 6.

Suppose that d = 2. Then by using Lemma 9.3.8 (i) we can assume that |U| = 2 and |V| = 0. From (ii) of Lemma 9.3.8 we can assume that U = {0,x} for some x ∈ K; x ≠ 0. But then Σu∈Uu = 0 + x = x and since V = Ø, condition (ii) of Definition 9.3.3 does not hold.

Suppose that d = 4. Then again from Lemma 9.3.8 (i) we can assume that either |U| = 4 and |V| = 0 or |U| = 2 and |V| = 2. In the former case, by Lemma 9.3.8 (ii) we can assume that U = {0,x,y,z} where x,y and z are distinct non-zero elements of Kr. Then condition (iii) of Definition 9.3.3 gives that

03+x 3+y 3+z 3+(0+x+y+z)3=0, so

(x+y)(x+z)(y+z)=0

which is impossible since x,y and z are distinct and non-zero. In the latter case, from Lemma 9.3.8 (ii) we can assume that U = {0,x} and that V = {y,z}, y ≠ z. Then from condition (iii) of Definition 9.3.3 we know that

03+x 3+(0+x)3=y 3+z 3.

But by Corollary 9.3.14 if y 3 = z 3 then y = z which is a contradiction.

To find a codeword of weight 6, for any distinct non-zero elements x,y and z of Kr, let w be the unique (by Corollary 9.3.14) element of Kr for which w 3 = x 3+y 3+z 3. Also, define u = w+x+y+z. Then w is not equal to x,y or z (since if w = x say, then w 3 = x 3, so 0 = y 3+z 3, so by Corollary 9.3.14 y = z) and u ≠ 0 (for w 3 + (x+y+z) = (x+y)(x+z)(y+z) ≠ 0, so by Corollary 9.3.14 w ≠ +x+y≠z). Now let U = {0,u} and V = {w,x,y,z}. Since u ≠ 0 and since w,x,y and z are distinct, [χ(U),χ(V)] is a word of weight 6 and it is easy to check that it is also a codeword in P(r).

Example 9.3.16 Construct K 3 as in Example 9.3.14. Following the notation of Theorem 9.3.15, let x,y, and z be distinct non-zero field elements, say x = β, y = β 3, and z = β 5. Then define
< previous page | page_218 | next page >


page_219

< previous page | page_219 | next page >

Page 219

so w = β 6. Now define u = w+x+y+z=β 6+β+β 4+β 5 = β 4. Then with U = {0,u} = {0,β 4} and V = {w,x,y,z} = {β 6,β,β 3,β 5}, we have that

[χ(U),χ(V)] = 00001001 01010110

is a codeword in P(3) of weight 6.

Exercises

9.3.17 Construct K 3 with 1+β+β 3 = 0. For the following elements x,y and z of K 3 define w and u as in Theorem 9.3.15 to construct a codeword of weight 6 in P(3).

(a) x = β, y = β 2, z = β 3

(b) x = β, y = β 4, z = β 6

(c) x = β 0, y = β 3, z = β 6

Theorem 9.3.18 P(r) is not a linear code.

Proof: As was noted at the beginning of the section, [χ(U),χ(V)]+[χ(A), χ(B)]=[χ(UΔA),χ(VΔB)]. From the proof of Theorem 9.3.15, we can construct codewords [χ(U),χ(V)] and [χ(A),χ(B)] of P(r) with U = {0,u 1}, V = {x,y 1,z 1,w 1} A = {0,u 2} and B = {x 2,y 2,z 2,w 2}. Then by Lemma 1.5, c = [χ(UΔA+u 1),χ(VΔB)] is codeword in P(r). Since |UΔA+u 1| ≤ 2, since the distance between c and [χ(UΔA),χ(VΔB)] is at most 2|UΔA+u 1| ≤ 4 and since P(r) has minimum distance 6, [χ(U),χ(V)]+[χ(A),χ(B)] is not a codeword in P(r). Therefore P(r) is a nonlinear code.

Being nonlinear, P(r) does not have a dimension, and we do not yet know the number of codewords in P(r) but this number will be obtained as a consequence of the encoding scheme.

9.4 Encoding extended Preparata codes

In Section 5.4 we saw that [image] is the generator of a 2 error-correcting BCH code with parity-check matrix

where β is a primitive element of GF(2r). Recall that deg(g(x)) = 2r. Since g(x) is the non-zero codeword of smallest degree, no linear combination of the first 2r

< previous page | page_219 | next page >


page_22

< previous page | page_22 | next page >

Page 22

only error pattern of weight 3, 111. The only error pattern not covered by Theorem 1.11.14 is 000. But by Exercise 1.11.5 we know that 000 is not detected.

Theorem 1.11.14 does not prevent a code C from detecting error patterns of weight d or greater. Indeed, C usually will detect some such error patterns.

Example 1.11.16 The code C = {001, 101, 110} has distance d = 1. Since d − 1 = 0, Theorem 1.11.14 does not help us determine which error patterns C will detect. But it does tell us that there is at least one error pattern of weight d = 1 which C will not detect. As we saw in Example 1.11.1, such an error pattern is 100. Note, however, that C does detect the error pattern 010 of weight d = 1.

Exercises

1.11.17 The code C = {0000, 1010, 0111} has distance d = 2. Using Exercise 1.11.5, show that the error pattern 1010 is not detected. Show that this is the only error pattern of weight 2 that C does not detect. Find all error patterns that C detects.

1.11.18 Find all error patterns which the code C 3 of Example 1.3.3 will detect. Note that C 3 is a single error-detecting code.

1.11.19 For each code C in Exercise 1.11.12 find the error patterns which Theorem 1.11.14 guarantees C will detect.

1.11.20 Let C be the code consisting of all words of length 4 which have even weight. Find the error patterns C detects.

1.12 Error-correcting codes

If a word v in a code C is transmitted over a BSC and if w is received resulting in the error pattern u = v+w, the IMLD correctly concludes that v was sent provided w is closer to v than to any other codeword. If this occurs every times the error pattern u occurs, regardless of which codeword is transmitted, then we say that C corrects the error pattern u. That is, a code C corrects the error pattern u if, for all v in C, v+u is closer to v than to any other word in C. Also, a code is said to be an t error-correcting code if it corrects all error patterns of weight at most t and does not correct at least one error pattern of weight t+1.

Example 1.12.1 Let C = {000, 111}.

(a) Take the error pattern u = 010. For v = 000,

d(000, v+u) = d(000, 010) = 1 and

d(111, v+u) = d(111, 010) = 2.
< previous page | page_22 | next page >


page_220

< previous page | page_220 | next page >

Page 220

rows of H in 9.1 can be zero. In fact, since g(x) generates a cyclic code, it follows that each submatrix of H formed by 2r consecutive rows has full rank, and so has an inverse. Define A to be the submatrix of H formed by the last 2r rows, and let H′ be formed by deleting the last 2r rows from H.

Example 9.4.1 Constructing K 3 using 1+x + x 3, we have that

Constructing GF(25) using 1+x 2+x 5 (see Exercise 5.1.15) we have that

Let m = mL, mR be any binary word of length 2r+1 − 2r − 2, where mL is a binary word of length 2r − 1 and mR is a binary word of length 2r − 2r − 1. Then by using polynomial notation for mL and mR, we have that

Now define

Theorem 9.4.2 Let r be odd. For any binary word m of length 2r+1 − 2r − 2, if χ(U) = [mL, pL] and if χ(V) = [mR, vR, pR], where pL and pR are paritycheck digits for mL and [mR, vR] respectively, then [χ(U),χ(V)] is a codeword in P(r).

Proof:
< previous page | page_220 | next page >


page_221

< previous page | page_221 | next page >

Page 221

But [image] Similarly, [image] and [image]. Therefore conditions (ii) and (iii) of Definition 9.3.3 hold, and clearly conditions (i) and (iv) are satisfied. Therefore [χ(U),χ(V)] is a codeword in P(r).

Corollary 9.4.3 P(r) has [image] codewords.

Proof: In Theorem 9.4.2 there are [image] choices for m, each giving a different codeword, the remaining digits of the codeword containing m being completely determined by conditions (i), (ii), and (iii) of Definition 9.3.3.

Algorithm 9.4.4 (encoding P(r)) Let mL and mR be words of length 2r−1 and 2r − 2r − 1 respectively. Let vR be as defined in Theorem 9.4.2. Then [mL, pL, mR, vR, pR] is the codeword corresponding to the message m = [mL, mR].

Example 9.4.5 Let r = 3, mL = 0110010 and mR = 1. Then

mL(β) = β+β 2+β 5 = β 0, mR(β) = β 0,

mL(β 3) = β 3+β 6+β 15 = β 2 and mL(β 3) = β 0.

From (2.3)

where A−1 was constructed in Example 9.4.1. Then we encode m = [0110010,1] to c = [mL, pL, mR, vR, pR] = [0110010, 1, 1, 111001, 1]. So in the notation used in Section 9.1, c = [χ(U),χ(V)], where

χ(U) = 01100101 and χ(V) = 11110011.

This is the codeword of P(3) considered in Example 9.3.4.

Exercises

9.4.6 Construct K 3 using 1+x + x 3. A−1 was constructed in Example 9.4.1. Encode the following messages using P(3).

(a) mL = 1010100 and mR = 1.

(b) mL = 1010100 and mR = 0.

(c) mL = 1111111 and mR = 1.

(d) mL = 1111111 and mR = 0.

(e) mL = 0000000 and mR = 1.

9.4.7 Construct K 5 using 1+x 2+x 5 (see Exercise 5.1.5). A−1 was constructed in Example 9.4.1 Encode the following messages using P(5).
< previous page | page_221 | next page >


page_222

< previous page | page_222 | next page >

Page 222

(a) mL = 10100...0 and mR = 000001000100...0.

(b) mL = 10100...0 and mR = 00...0.

(c) mL = 10100...0 and mR = 11110...0.

(d) mL = 00...0 and mR = 100...0.

9.4.8 In Exercise 9.4.7, what is the length of

(a) mL?

(b) mR?

9.5 Decoding extended Preparata codes

From Theorem 9.3.15, P(r) has minimum distance 6 so we want an algorithm that corrects up to 2 errors. Let w be a received word and write w = [wL, pL, wR, pR] where wL and wR are both words of length 2r − 1 and where pL and pR are the parity check digits. Then we can calculate [wL(β),wL(β3)] = wL H and [wR(β),wR(β3)] ↔ wR(H). We consider various cases depending on where the errors occur.

1. If errors are confined to the parity check digits then

(by Definition 9.3.3 (ii) and (iii)), so this case is easily checked.

2. If there are no errors in wL, one error in position i of wR and at most one error in the parity check digits, then

(by Definition 9.3.3 (ii) and (iii)). If this last equation holds then write βi = wL(β)+wR(β) and change the ith digit in wR and at most one parity check digit.

3. If there are no errors in wR, one error in position i of wL and at most one error in the parity check digits, then using Lemma 9.3.8(i) we can repeat the steps in case 2 above to find that (wR(β)+wL(β))3 = wR(β 3)+wR(β))3+wL(β 3); in this case write β i = wR(β)+wL(β), change the ith digit of wL and at most one parity check digit.

4. If two errors occur in wR, say in positions i and j, then again by Definition 9.3.3
< previous page | page_222 | next page >


page_223

< previous page | page_223 | next page >

Page 223

so βi + β j and β 3i + β 3j are known; i and j can be found in the same manner as was used for the 2 error-correcting BCH code (see Section 5.5).

5. If two errors occur in wL then, as in case 3, we can use Lemma 9.3.8(i) and the argument in case 4 to find the locations of the errors.

6. If there is one error in wL and one error in wR, say in positions i and j respectively, then again by Definition 9.3.3

We can solve these two equations for βi and βj as follows. From the first equation,

βj = wL(β)+βi+wR(β).

Substituting this into the second equation gives

Simplifying this gives

so

Therefore

So in all cases we can easily calculate the locations of the errors. The parity check conditions on each half of w make it easy to decide which of the above cases applies to w. Putting all of these observations together we get the following algorithm. The steps in the algorithm correspond to the cases just considered.
< previous page | page_223 | next page >


page_224

< previous page | page_224 | next page >

Page 224

Algorithm 9.5.1 (decoding P(r)) Let w = [wL,PL, wR, pR] be a received word.

0. Calculate L 1 = wL(β), L 3 = wL(β 3), R 1 = wR(β) and R 3 = wR(β 3).

1. If L 1+R 1 = 0 and [image] then the only errors occur in the parity check digits.

2. If [image] then write βi = L 1+R 1. Correct position i of wR and at most one parity check digit; ask for retransmission if both parity check digits need to be changed.

3. If [image] then write βi = L 1+R 1. Correct position i of wL and at most one parity check digit; ask for retransmission if both parity check digits need to be changed.

4. If both halves of w have even parity and

for some i and j, then correct positions i and j of wL.

5. If both halves of w have even parity and

for some i and j then correct positions i and j of wR.

6. If both halves of w have odd parity, then write [image] and [image] Correct position i of wL and position j of wR.

7. If no closest codewords has yet been found then conclude that at least three errors occurred during transmission, and ask for a retransmission.

Example 9.5.2 Decode the following received words which were encoded using P(3), where GF(23) is constructed using 1+x + x 3.

(a) 10010011 11100111

(b) 10100100 10001001

(c) 10001000 11101001

Decoding (a):

(0) [L 1, L 3] = wL H = [111, 110] and [R 1,R 3] = wR H = [101, 110].

(1) L 1+R 1 = 111+101 = β ≠ 0.

(2)

(3)

< previous page | page_224 | next page >


page_225

< previous page | page_225 | next page >

Page 225

(4) x 2+βx+(β 3+β 15+β 3+β 3+β 3)/β = x 2+βx+β 6 = (x+β 2)(x+β 4).

Decode w to 10010011 11001111.

Decoding (b):

(0) [L 1,L 3] = wL H = [010, 011] and [R 1,R 3] = wR H = [111, 011].

(1) L 1+R 1 = 010+111 = β 6 ≠ 0.

(2)

(3)

(4) and (5) Both halves of w have odd parity.

(6)

so i = 0. Then we can immediately write

so j = 2. Decode w to 00100100 10101001.

Decoding (c):

(0) [L 1,L 3] = wL H = [111, 011] and [R 1,R 3] = wR H = [100, 000].

(1) L 1+R 1 = 11+100 = β 4 ≠ 0.

(2)

(3)

Let βi = L 1+R 1 = β 4, so i = 4. However, changing position 4 of wL requires both parity check digits to be changed, so we ask for retransmission (since we can find a codeword distance 3 from w).

Exercises

9.5.3 Decode the following received words that were encoded using P(3), where GF(23) is constructed using 1+x + x 3.

(a) 10000001, 11101000

(b) 00011010, 01000010

(c) 00100101, 10100100

(d) 01010110, 00011110
< previous page | page_225 | next page >


page_226

< previous page | page_226 | next page >

Page 226

(e) 11101000, 10001001

(f) 10011001, 01010101

(g) 01000111, 11001000

(h) 10101101, 11010000

(i) 11101110, 01010101

(j) 10111011, 01101010

(k) 01011101, 11101101

(l) 10011100, 10100100

(m) 01101101, 10011000

(n) 10101010, 10111011

(o) 10100101, 00010001

9.5.4 Decode the following received words that were encoded using P(5), where GF(25) is constructed using 1+x 2+x 5 (see Exercise 5.1.15).

(a) 11000 11000 10000 00000 00000 10000 10, 00011 11000 00000 00000 00011 00100 00

(b) 10100 00000 10000 00000 00000 00000 00, 00000 10001 00000 00100 01010 10111 00

9.5.5 If w is a received word, the left half of which has odd parity and the right half has even parity, can w ever be decoded to a codeword distance at most 2 from w at Step 2 of Algorithm 9.5.1?
< previous page | page_226 | next page >


page_227

< previous page | page_227 | next page >

Page 227

Chapter 10
Classical Cryptography

Cryptography is about communication in the presence of adversaries.1 The best-known example is that of confidentiality or privacy—maintaining secrecy in communications over an unsecured channel. The cryptographic solution is encryption, which modifies the data in an attempt to render the communication unintelligible to all but the intended recipients. Cryptography can be regarded as the science of mathematical techniques for protecting data (from malicious or unauthorized actions) by transforming the data itself.

In addition to confidentiality, cryptography can be used to satisfy various information security objectives collectively known as authentication. Data integrity provides detection of data manipulation, including alteration, delay, and replay of messages. Message (or data origin) authentication provides proof of the source of the information. Identification corroborates the identity of an entity. Non-repudiation is a service which prevents entities from denying previous commitments. A digital analogue of the hand-written signature, specific to the message, is fundamental to many authentication schemes.

Cryptography is closely connected with cryptanalysis, the study of mathematical and other techniques for defeating information security objectives; together, these areas are known as cryptology. Cryptanalysis under various assumptions concerning the capabilities of the adversary is an essential part of applied cryptology, providing some measure of confidence in schemes for which mathematical proof of security is not available.

Section 10.1 establishes the basic framework used in Chapters 10–12. In classical cryptography, privacy of communications depends on a secret shared between the communicating parties; Section 10.2 examines a few such symmetric-key schemes. The one-time pad is of particular interest, as an example of a simple scheme that cannot be broken regardless of the computing power of the adversary. Unfortunately, the one-time pad is rather unwieldy in practice due to the amount of key material required. Section 10.3 discusses the Data Encryption Standard, the

1Ronald L. Rivest, from [71].
< previous page | page_227 | next page >


page_228

< previous page | page_228 | next page >

Page 228

best-known symmetric-key scheme in history. Rather than the unconditional security of the one-time pad, DES was designed so that it would be ''computationally infeasible" for an adversary to break the scheme.

A basic symmetric-key property is that the secret in the scheme must be shared with correspondents—the ability to create a secret message cannot be separated from the ability to interpret such a message. A landmark 1976 paper by Diffie and Hellman [27] introduced the concept of public-key cryptography; the separation of creation and interpretation capabilities is a consequence. Security relies on the apparent intractability of various computational problems. Chapter 11 contains an overview of several number-theoretic topics fundamental to public-key schemes. Chapter 12 discusses public-key concepts and methods.

10.1 Encryption schemes

The following framework will be used in the study of cryptographic tools or primitives.

• An alphabet [image] is a finite set.

• A message space [image] over [image] consists of strings of symbols from the alphabet. For example, if [image] = {0, 1} then messages consist of strings of 0s and 1s; the set of messages of length n is denoted {0, 1}n, and the set {0, 1}* consists of all messages of finite length.

• An encryption scheme or cipher consists of message spaces [image] and C, a set [image] (known as the key space), and functions Ek : → C, Dk : C → [image] for k ∈ [image] with Dk(Ek(m)) = m for all m ∈ [image]. Ek is called an encryption function, and Dk is the corresponding decryption function. In this setting, elements of [image] are plaintext, C is called a ciphertext space, and elements of C are ciphertext.

• In some schemes, keys are written as ordered pairs k = (e, d) where e is used for encryption and d is used for decryption. In this context, (e, d) is called a key-pair, and we will write Ee = Ek and Dd = Dk.

Figure 10.1 is a schematic of the basic scenario where Alice wishes to pass a private message m to Bob in the presence of an adversary Eve. In traditional symmetric-key ciphers (see Section 10.2), the secure channel is required in order to pass the keys themselves, and may represent a trusted courier or other reliable method for maintaining privacy. In public-key schemes (see Chapter 12), the channel represents a method of verifying the authenticity of the public portion of a key.

Adversaries may be classified by capability. A passive adversary is limited to eavesdropping on the unsecured portion of the conversation between Alice and Bob. An active adversary, in addition to eavesdropping, may alter or inject messages, or perhaps even block transmission entirely. Cryptography can hope to pro-
< previous page | page_228 | next page >


page_229

< previous page | page_229 | next page >

Page 229

Figure 10.1: Communication using encryption.

vide secrecy, and detect altered or forged messages. Delivery of messages in the face of a sufficiently powerful adversary cannot be guaranteed, although encrypted messages may be sent periodically to discover communication disruptions.

Informally, an encryption system is considered breakable if an adversary can systematically recover plaintext from ciphertext (or even worse, the key can be computed). Three kinds of security are noted:

• The system is unconditionally secure if the adversary can gain no knowledge about plaintext (except possibly its length), regardless of the amount of ciphertext available and the computational resources available to the adversary.

• A system is computationally secure if breaking it is ''computationally infeasible" under some prescribed level of available computing resources, and using the best known cryptanalytic techniques.

• If breaking is provably at least as difficult as solving a mathematical problem believed to be intractable, the system is said to be provably secure.

These intuitive notions of security will be made somewhat more precise in the following sections.

In 1883, Auguste Kerckhoffs published what Kahn calls the second great book in cryptology of the "outward-looking kind."2 Several fundamental principles for selecting ciphers were listed: the system should be, if not theoretically unbreakable, unbreakable in practice; compromise of the system should not inconvenience the correspondents; the key should be rememberable without notes and should be easily changeable; the cryptograms should be transmissible by telegraph; the apparatus or documents should be portable and operable by a single person; the system should be easy, neither requiring knowledge of a long list of rules nor involving mental strain. The second of these is sometimes called Kerckhoffs' prin-

2Kahn [48] credits Naples-born Giovanni Battista Porta as "the first to delineate a coherent image of cryptology" in a 1563 work, but whose "views no longer sufficed after the invention of the telegraph."
< previous page | page_229 | next page >


page_23

< previous page | page_23 | next page >

Page 23

And for v = 111,

d(000, v+u) = d(000, 101) = 2

d(111, v+u) = d(111, 101) = 1.

Thus C corrects the error pattern 010.

(b) Now take the error pattern u = 110. For v = 000,

d(000, v+u) = d(000, 110) = 2 and

d(111, v+u) = d(111, 110) = 1.

Since v+u is not closer to v = 000 than to 111, C does not correct the error pattern 110.

The IMLD table can be used to determine which error patterns a code C will correct. In each error pattern column of the table, all possible error patterns (which means each word in Kn) occurs once and only once (for if the error pattern u occurs twice in a column for some codeword v, then u occurs in rows corresponding to distinct received words, say w 1 and w 2; thus u = v+w 1 = v+w 2, which is impossible for [image] Also, an asterisk is placed beside the error pattern u in the column corresponding to a codeword v in the IMLD table precisely when v+u is closer to v than it is to any other codeword. Therefore an error pattern u is corrected if an asterisk is placed beside u in every column of the IMLD table.

Example 1.12.2 For the code C = {000, 111}, the IMLD table is in Table 1.1. In every row of the table where the error pattern 010 occurs (rows 3 and 6), IMLD correctly concludes which word v was sent. Also, in at least one row (row 4) where the error pattern 110 occurs, if 111 is sent and 001 is received, IMLD incorrectly concludes that 000 was sent. Note that this code corrects the error patterns 000, 100, 010, and 001 which receive an asterisk each time they occur. So C is a 1 error correcting code.

Example 1.12.3 Let C = {0000, 1010, 0111}. The IMLD table for C is Table 1.2. The code C will not correct the error pattern u = 1010. This error pattern occurs in the rows where w = 0000, 1010, and 1101. In only one case, where w = 1101, does IMLD correctly conclude which word v was sent. Note that the error pattern u = 1010 receives an asterisk only in the column for v = 0111 and not in the other two columns. C does correct the error patterns 0000, 0100 and 0001.

Example 1.12.4 Let C = {001, 101, 110}. Does C correct the error pattern u = 100? We construct only the three rows of the IMLD table where 100 appears. Since u = v+w and we know u and v, we can find the received words from w = u+v. Notice that u = 100 does not receive an asterisk in every column in the following table, so C does not correct 100.
< previous page | page_23 | next page >


page_230

< previous page | page_230 | next page >

Page 230

ciple, and says that security of the cipher should rest with the keys alone; that is, security is maintained even in the face of an adversary in possession of the encryption scheme.

Attacks on the communication of Figure 10.1 typically have the objective of recovering plaintext from ciphertext, or even the key itself, although other more limited information (such as particular plaintext bits) may be targeted. Traffic analysis attempts to learn something about the messages by examination of the pattern of transmission. For example, a sudden flurry of exchanges may be notable even if the underlying messages cannot be discovered. We are mainly interested in attacks against the encryption scheme itself.

1. In a ciphertext-only attack, the adversary attempts to recover plaintext or the key by observing some amount of ciphertext. A scheme vulnerable to this kind of attack is considered completely insecure.

2. In a known-plaintext attack, the adversary is in possession of some quantity of plaintext and corresponding ciphertext. The typical case is that the plaintext and ciphertext pairs correspond to some (unknown) key, and the goal is to recover the key or decrypt some additional ciphertext which is believed to have been encrypted with the same key.

3. In a chosen-plaintext attack, ciphertext corresponding to plaintext selected by the adversary is available. A possible scenario is that the adversary has obtained temporary access to the equipment (but not the key) used for encryption. If the ciphertext may be chosen interactively based on previous results, the attack is called adaptive.

4. In a chosen-ciphertext attack, plaintext corresponding to ciphertext selected by the adversary is available.

Other possibilities include implementation-specific and tempest attacks which exploit device properties; timing of computations and power consumption, for example, have been used against certain systems [53, 52]. Bribery, blackmail, and other ''give-me-the-key" attacks have been historically rather successful.

10.2 Symmetric-key encryption

Many classical ciphers have the property that a "secret" is shared between parties, and knowledge of this information allows both parties to encrypt and decrypt. More precisely, an encryption scheme is said to be symmetric-key if finding Dd from e and Ee from d require roughly the same resources.

A simple substitution cipher consists of 1−1 mappings k on the source alphabet. Encryption is performed by applying k to each symbol in the message; i.e., if m = m 0 m 1 ..., where mi are symbols from the alphabet, then Ek(m) = Ek(m 0 m 1 ...) = k(m 0)k(m 1) ....
< previous page | page_230 | next page >


page_231

< previous page | page_231 | next page >

Page 231

Example 10.2.1 The shift cipher is a special case of the substitution cipher. The key k is a fixed shift 0 ≤ k < n on the symbols {a 0,..., an−1} of the alphabet, mapping aj [image] a(j+k)modn.3 The popular rot13 encryption function corresponds to k = 13 on the source alphabet {a, ..., z}, rotating by 13 positions; i.e., the letters are numbered 0–25, and rot13 adds 13 to each source letter, reducing the result modulo 26.4 As a visual clue, lowercase letters will be used for plaintext, and uppercase for ciphertext. For example, since

rot13(rotate) = rot13(17, 14, 19, 0, 19, 4) = (4, 1, 6, 13, 6, 17) = EBGNGR,

the plaintext 'rotate' is encrypted to ciphertext 'EBGNGR'. This particular shift also has the property that rot13(rot13(m)) = m; i.e., encryption and decryption are the same function. Julius Caesar is reputed to have used a shift cipher with k = 3.

The shift cipher is completely insecure against a chosen-plaintext attack. In fact, a ciphertext-only attack is easily mounted, since the 26 possible keys can be examined exhaustively.

A simple substitution cipher may resist exhaustive search of the keyspace, even if the source alphabet is small. However, frequency analysis may be effective if the underlying source text characteristics are known. For example, Figure 10.2 lists single-character frequencies obtained from a modest sampling of English-language newspapers and novels [3]. Ciphertext from simple substitution respects the frequency distribution of the source. If the source statistics are roughly according to Figure 10.2, for example, then the most common ciphertext symbol probably corresponds to 'e'. Similarly, the symbols {j, q, x, z} are significantly less frequent in this kind of source, and likely correspond to less frequent ciphertext symbols. Statistics from digrams (pairs of letters) and trigrams (triples of letters) can be used in a similar fashion.

Example 10.2.2 Consider the case that the alphabet is the 26 letters a−z, and the substitution cipher is a permutation on these letters, perhaps

where the source letters in the first row are mapped to the ciphertext letters in the second row; e.g., Ek(trek) = MJXE. This particular permutation is easy to remember, since it is constructed by placing the keyword 'BIKE' under the letter 'h' and then filling in the rest of the letters in order.

3The ''mod" notation is reviewed in Chapter 11; here, "(j+k) mod n" adds integers j and k as usual, subtracting n if the result exceeds n − 1.

4The source documentation with the GNU Emacs editor notes that "Rot13 encryption is sometimes used on USENET...for material some might consider offensive."
< previous page | page_231 | next page >


page_232

< previous page | page_232 | next page >

Page 232

Figure 10.2: Frequencies of single characters in English text.

The scheme is completely insecure against a chosen-plaintext attack (since the key can be recovered by encrypting the message m ab...z). Having only ciphertext available makes things a little more difficult. Although the exhaustive attack of trying each possible key in turn may be infeasible (since the keyspace is of size 26!≈4.1026, the scheme may be subject to frequency analysis.

A simple substitution cipher is sometimes called a monoalphabetic substitution cipher. Attacks based on frequency analysis become infeasible if the message space is over some large alphabet. Small alphabets, such as that in Example 10.2.2, can be expanded by combining several symbols and forming a message space over this much larger alphabet. The Data Encryption Standard (DES, to be discussed in Section 10.3.2) in its basic form is a substitution cipher over a source alphabet of size 264.

Polyalphabetic ciphers attempt to hide the original source frequencies by mapping each source symbol to one of several possible ciphertext symbols. The Vigenère cipher is a well-known example. Symbols from the source alphabet [image] are identified with integers in [0,| [image] |), and a keyword k = k 0...k n−1 is selected for some ki ∈ [image]. Encryption is performed on n-character blocks of the source by character-wise addition modulo | [image] | with the keyword.

As an example, suppose [image] consists of the symbols a-z numbered 0–25. The following table illustrates the procedure on a short plaintext using key 'KEY'
< previous page | page_232 | next page >


page_233

< previous page | page_233 | next page >

Page 233

where the addition is modulo 26. Decryption is performed by subtracting modulo 26 the keyword from the ciphertext.

The simple analysis based on source frequency characteristics is partially thwarted by the mapping of alphabet symbols to different ciphertext letters, depending on the position in the source; e.g., the source letter 'e' is mapped onto the set {C,I,O} in the above example. However, if the keylength ℓ can be discovered, then monoalphabetic frequency analysis can be used on the submessages formed from letters that are a multiple of ℓ apart in the ciphertext. In the above fragment, the submessage formed from the ciphertext in positions 0,3,6,..., for example, is obtained by adding (modulo 26) the first letter of the 3-letter key to plaintext letters from the corresponding positions.

In 1863, Friedrich Kasiski proposed a method (now known as a Kasiski examination) to recover the keylength via analysis of the distances separating identical fragments of ciphertext.5 Occasionally, identical fragments of source text will align with the same portion of the key, resulting in identical portions of ciphertext. In the example, this occurs with the 4-character fragment 'ells'. Typically, there will also be ''accidental" matches—identical fragments of ciphertext which did not result from identical fragments of plaintext—although these tend to be less likely with longer match lengths. In the case of a "true" match, the keylength divides the difference in positions. The possibility of accidental matches means that it may not be enough to look only at the greatest common divisor of the distances, and in general the larger and more common factors are examined first as keylength candidates.

Example 10.2.3 A Vigenère cipher has been used on "typical" English source text which was first processed to remove punctuation and blanks so that the source consists only of the 26 letters a-z. The following ciphertext was produced:

The spaces in the ciphertext were added for readability, and are not represented in the offsets.

5Kahn [48] writes that Kasiski died in 1881 "almost certainly without realizing that he had wrought a revolution in cryptology."
< previous page | page_233 | next page >


page_234

< previous page | page_234 | next page >

Page 234

The 3-character fragment 'ZQH' appears in several places, three of which are underlined. The corresponding offsets are 110, 138, and 226, respectively. It is likely that the keyword length ℓ divides the difference of any two of these offsets. The differences 138 − 110 = 28 = 227 and 226 − 110 = 2229 suggest ℓ| (227, 2229) = 22. The case ℓ = 1 corresponds to a simple shift.

At this stage, the cryptanalyst may examine the frequency distributions for each candidate keylength, looking for those which have the characteristics of the expected source language. The frequencies are tabulated corresponding to each position of the key; e.g., for the candidate keylength ℓ = 2, the frequencies are tabulated for the submessage formed from the characters in the even offsets, then from the odd offsets. This gives, for example,

For a given source distribution, we expect that each line for the correct keylength should reflect the source frequencies. In this example, the frequencies suggest that ℓ = 4 is more likely than ℓ = 2, although the limited amount of ciphertext hinders the analysis.

In short, the analysis so far suggests that ℓ = 4 is a likely candidate. The frequency analysis of Figure 10.2 shows that 'e' is significantly more prevalent than any other letter in typical English text. We suspect that one of the higher-frequency ciphertext letters, in each of the ℓ = 4 lines, corresponds to plaintext letter 'e'. In this particular example, choosing 3 or 4 (or perhaps a few more) ciphertext letters from each of the lines in the ℓ = 4 case appears likely to yield the mapping from plaintext letter 'e'.

The following table lists the 4 most frequent ciphertext letters corresponding to the higher frequencies in each line of the ℓ = 4 case, along with the appropriate key letters which map 'e' to each of these ciphertext letters:

At this stage, an exhaustive search using keywords built from the likely key letters could be performed by machine. In the example, this requires 44 = 256 decryptions (using keywords such as 'BXRG').
< previous page | page_234 | next page >


page_235

< previous page | page_235 | next page >

Page 235

If the keyword was chosen from a dictionary, then it may be especially easy to locate. In the example, this restriction leads to words such as 'LEND' and 'BIRD', and the second of these yields:

which appears to be English text. The complete decryption, with punctuation and case restored, is:

The water of the Gulf stretched out before her, gleaming with the million lights of the sun. The voice of the sea is seductive, never ceasing, whispering, clamoring, murmuring, inviting the soul to wander in abysses of solitude. All along the white beach, up and down, there was no living thing in sight. A bird with a broken wing was beating the air above, reeling, fluttering, circling disabled down, down to the water.6

The 'ZQH' ciphertext fragments used in the Kasiski test correspond to plaintext 'ing'. Longer repeated fragments, such as 'NLUP' (corresponding to 'with') and 'ZQHBY' (corresponding to 'ingth'), each appearing twice, could have been chosen. We conveniently avoided fragments such as 'PVZ' (appearing at positions 1 and 135) which are accidental matches and correspond to 'hew' and 'mur', respectively. These fragments suggest keylengths dividing 134, which would have been incorrect.

More challenging examples of cryptanalysis of a Vigenère cipher may be found in Kahn [48] and Stinson [86]. In particular, the indices of coincidence defined in [86] provide a more methodical approach to finding the keylength and the keyword candidates.

The simple substitution and Vigenère ciphers are examples of block ciphers, since they transform a message by applying a fixed mapping to blocks consisting of a fixed number of characters. They are memoryless in the sense that the mapping of a block does not depend on its position in the message. Stream ciphers allow the mapping to depend on the position within the message, and are sometimes called state ciphers.

Encryption schemes are typically defined for a binary alphabet [image] = {0, 1}, and block (and stream) ciphers are defined on some fixed number of bits, called the blocklength. Messages over an alphabet [image] = {a 0,...,an−1} can be regarded as written over a binary alphabet by associating aj ↔ j and writing j as a binary number with ⌈log2 n [image] bits. In the case of a 26-letter alphabet as in Example 10.2.2, each letter is replaced by a 5-digit binary number, with 'g' ↔ 6 = 00110, for example. The encryption and decryption functions in these examples are then considered to operate on blocks of length 5.

6From The Awakening by Kate Chopin.
< previous page | page_235 | next page >


page_236

< previous page | page_236 | next page >

Page 236

For messages m and m′ over {0, 1} of the same length, bitwise addition modulo 2 will be written m ⊕ m′; e.g., 1100⊕1010 = 0110. Programmers know this operation as ''exclusive or," sometimes written XOR, and amounts to the schoolchild's paradise where addition is never complicated by carry.

Example 10.2.4 The Vernam cipher is a stream cipher on [image] = {0, 1}. Keys also consist of messages over [image], and encryption maps m [image] c = m ⊕ k. By the usual properties of modular arithmetic, it is easy to see that the plaintext is recovered from c by c ⊕ k = m ⊕ k ⊕ k = m ⊕ 0 = m; i.e., decryption is the same as encryption.

If the key digits (known as the keystream) are the result of independent Bernoulli trials with probability 1/2 (e.g., independent tosses of a fair coin) and are used only once, then the cipher is known as a one-time pad, and is unconditionally secure against a ciphertext-only attack. The intuitive idea is that every message (of the same length as the ciphertext) maps to the ciphertext for some key, and there is no reason to favor one key over another. Shannon's work in the 1940s [80] made these notions precise, showing that, roughly speaking, any unconditionally secure symmetric-key encryption scheme requires a key as long as the message. (A more precise statement on the requirements for keys can be given in terms of the entropy associated with the message space.)

In some early applications of the Vernam cipher, the key information was written on notepads and pages were discarded after a single use, leading to the name "one-time pad." Ciphertext obtained from the same key would enable an adversary to learn something about the underlying messages; e.g., if c = m ⊕ k and c′ = m′ ⊕ k then

c ⊕ c′ = (m ⊕ k) ⊕ (m′ ⊕ k) = m ⊕ m′

which may allow cryptanalysis (in particular, m = m′ is immediately discovered).

The Codebreakers has a few amusing accounts of early attempts to manage the quantity of key material required, and the problem continues to limit the application of the one-time pad. According to Peter Wright of British Intelligence [101], the Russians ran short of key material during the early years of WWII, and duplicate sets of the pads were distributed to different embassies in the West. The "VENONA codebreak" exploited this misuse of the scheme, examining vast amounts of ciphertext for matches across several channels used by the Russian government.7

Widely-used ciphers such as DES sacrifice the unconditional security of the one-time pad for more manageable schemes that are hoped to be computationally

7As part of a series on the "Cold War Experience" in 1998, the Cable News Network (CNN) reported that "The VENONA project at the U.S. National Security Agency cracked many supposedly secure Soviet messages, including some that employed one-time pads. The project lasted from 1943 to 1980, and some of its records were declassified in 1995." Unfortunately, CNN neglected to note that the scheme failed the fundamental single-use requirement, and claimed that the one-time pad "can also be cracked by painstaking trial and error, trying all of the millions of possible combinations."
< previous page | page_236 | next page >


page_237

< previous page | page_237 | next page >

Page 237

secure. The requirement of randomly-generated keys should also be noted. Common ''random number generators" produce a sequence of numbers completely determined by an initial seed chosen from a finite set. Although they possess many of the features useful in simulations, they are not random in the sense of unpredictability required in cryptography.8.

Exercises

10.2.5 The following ciphertext was obtained under a Vigenère cipher of the type discussed in Example 10.2.3:

Recover the key word of the cipher, given that the underlined triple of letters represents a very common three-letter word, and that the overscored letters 'AE' represents the combination 'an' at the beginning of a three-letter word.

10.2.6 Write a program to implement a substitution cipher of the form in Example 10.2.2 (the authors used awk). The program should accept a keyword and alignment position as arguments. Encrypt a sample of at least 300 characters of English-language text, and then perform a cryptanalysis using frequency information and knowledge of the scheme used (but proceed as if the keyword itself is unknown).

10.2.7 This exercise follows the outline given in Example 10.2.3, providing sufficient information to recover the plaintext without the use of a computer. Readers with access to software tools may wish to develop a more realistic exercise.

The following intercepted ciphertext is known to have been produced by a Vigenère cipher:

8Goldberg and Wagner [39] discovered that random number generation in early versions of the Netscape web browser was significantly more predictable than advertised, compromising security of transactions. The episode was rather embarrassing for Netscape, which had refused to allow public scrutiny of its algorithms.
< previous page | page_237 | next page >


page_238

< previous page | page_238 | next page >

Page 238

Repeated fragments of 'MHH' are located at positions 74, 179, and 404 (other choices are possible, including the longer fragment 'SRHU').

(a) If ℓ is the length of the (unknown) key, then we may suspect that ℓ | 179 − 74 = 3.5.7 and ℓ | 404 − 179 = 3252. If these repeated fragments of ciphertext align with identical portions of plaintext, show that ℓ ∈ {1, 3, 5, 15}.

(b) Suppose the candidate ℓ = 15 is considered unlikely. The relative frequencies (from 830 total ciphertext letters) for the remaining candidates, along with corresponding ciphertext letters, are given by:

Explain why this summary suggests that ℓ = 5 is more likely than the others, under the assumption that the original message has single-letter frequencies corresponding roughly to Figure 10.2.

(c) Find the corresponding key letter candidates for several of the popular ciphertext letters in the case ℓ = 5, where the mapping is from the distinguished plaintext letter 'e'.

(d) It is suspected that the correspondents choose keywords from a dictionary. Find the keyword, and decrypt at least part of the ciphertext. It may be helpful to note that some of the submessages contain a distinguished (high-frequency) ciphertext letter.

10.2.8 Data compression removes some redundancy in the data. If compression is to be used in conjunction with encryption, does it make more sense to compress and then encrypt, or vice versa?

10.3 Feistel ciphers and DES

This section examines in brief a class of ciphers which includes DES, the best-known symmetric-key cipher. We're mainly interested in the ''structural" features

< previous page | page_238 | next page >


page_239

< previous page | page_239 | next page >

Page 239

and vulnerabilities of these ciphers; readers interested in the details of DES should consult the references listed in Section 10.4.

Two ciphers are considered, the New Data Seal (NDS) and the Data Encryption Standard (DES), each built on a Feistel cipher. Feistel ciphers are block ciphers, although a larger framework (such as DES) is commonly used in modes in which the overall scheme is a stream cipher. Among typical design criteria for block ciphers, the most fundamental is the requirement that the keyspace be sufficiently large so that exhaustive search is infeasible. Similarly, the blocksize should be large enough to frustrate the building of a significant database from observed ciphertext. The goal of confusion is to complicate the relationship between key and ciphertext, and diffusion attempts to spread the structure present in the plaintext so that a ciphertext bit depends on many plaintext bits. Efficiency in terms of speed, memory, and other computer resources may be essential, and the scheme may need to be suitable for hardware implementation.

A Feistel cipher maps messages of length 2n bits to ciphertext of the same length, where [image] The input to the scheme is the message itself and a key k. Encryption proceeds through a series of r rounds.

• A key-scheduling algorithm determines subkeys kl,...,kr from a specified key k. Each subkey determines a function

• A 2n-bit message m is split into n-bit left and right halves, written m = (m 0, m 1). The rounds can be written

The output interchanges the left and right halves of the last round, giving c = (mr+1, mr).

• The interchange allows decryption to use the same form of the r rounds, but with the subkeys in reverse order. To see this, define cj = mr+1−j. Then c = (c0, c1) and the assignment in round r is easily solved for mr−1, giving

which is of the form found in round 1.
< previous page | page_239 | next page >


page_24

< previous page | page_24 | next page >

Page 24

Received | Error Pattern |  |  | Decode
w | 001+w | 101+w | 110+w | v
101 | 100 | 000* | 011 | 101
001 | 000* | 100 | 111 | 001
010 | 011 | 111 | 100* | 110

Exercises

1.12.5 Let C = {001, 101, 110}. Does C correct the error pattern u = 100? What about u = 000?

1.12.6 Prove that the same error pattern cannot occur more than once in a given row of an IMLD table.

1.12.7 Prove that the zero error pattern is always corrected.

1.12.8 Which error patterns will the code C = Kn correct?

The distance of a code can be used to devise a test for error-correcting which avoids at least some of the manual checking from the IMLD table. The next theorem gives the test. Recall that the symbol [image] x [image] denotes the greatest integer less than or equal to the real number x. For example, [image] 5/2 [image] = 2, [image] 3 [image] = 3, and [image] 1/2 [image] = 0.

Theorem 1.12.9 A code of distance d will correct all error patterns of weight less than or equal to [image]. Moreover, there is at least one error pattern of weight 1 + [image] (d − 1)/2 [image] which C will not correct.

Proof: Let u be an error pattern of weight wt(u) ≤ (d − 1)/2. Let v and w be codewords in C with w ≠ v. We want to show that d(v,v+u) < d(w,v+u).

since wt(u) = d(v+u,v), and 2wt(u) + 1 ≤ d.

Therefore C corrects u. Now let v and w be codewords with d(v,w) = d. Form an error pattern u by changing d − 1 − [image] of the d 1's in v+w to 0's. Then
< previous page | page_24 | next page >


page_240

< previous page | page_240 | next page >

Page 240

The use of rounds in this fashion is attractive, as it offers the possibility of using relatively simple (if not elegant) mappings at each stage, reducing resource requirements. Multiple rounds (r = 16 in DES) add confusion and diffusion. The keyspace, of course, must be sufficiently large to prevent exhaustive search under some specified level of computing ability granted to the adversary.

10.3.1 The New Data Seal

NDS is a relatively simple Feistel cipher, in part because the key schedule consists of a single key. Unfortunately, this turns out to be an Achilles' heel, leading to a chosen-plaintext attack considered in this section.

We consider the case that n = 64 (so that messages are 2n = 128 bits in length) and r = 16 rounds. The key is a mapping k: {0, 1}8 → {0, 1}8. The necessary (but not sufficient) requirement of a keyspace sufficiently large to prevent exhaustive search is clearly satisfied, as the number of such keys is

The system contains two fixed (non-secret) maps, S 0, S 1 : {0, 1}4 → {0, 1}4, and the key schedule consists of the single key k, used on each round. To compute fk(mi) for a 64-bit half-message mi:

1. Break mi into eight 8-bit bytes, and let [image] denote the byte formed from the first bit of each byte of mi.

2. Break each byte of mi into two 4-bit nibbles, applying S 0 to the left half and S 1 to the right half.

3. Interchange the nibbles of the jth byte of the S 0 S 1-output in the case that the jth bit of k( [image]) is 1.

4. A fixed (non-secret) permutation is applied to the 64 bits of output.

A schematic for the round appears in Figure 10.3. The final permutation prevents a reduction to eight independent smaller schemes.

Chosen-plaintext attack on NDS

The use of the same key in every round of NDS is a weakness which can be exploited in a chosen-plaintext attack. The key itself can be recovered, completely breaking the scheme.

Let T = Tk be the transformation corresponding to a single round of NDS, i.e., T(mi−1, mi) = Tk(mi−1, mi) = (mi, mi−1 ⊕ fk(mi)), and let F = T16 denote all 16 rounds. The essential observation is that F and T commute: FT(m) = T 16 T(m) = TT 16(m) = T F(m).

Under Kerckhoffs' principle, we assume that the adversary knows everything about the scheme except the key k. Fix q ∈ {0, 1}8. The key is recovered if we can find k(q) for each such q. The attack proceeds as follows.
< previous page | page_240 | next page >


page_241

< previous page | page_241 | next page >

Page 241

Figure 10.3: The function f in one round of NDS.

1. Embed q in a message m = (m 0,m 1) so that [image]. The adversary obtains the ciphertext (m 16,m 17) = F(m) corresponding to the chosen-text m.

2. Let [image] denote one of the 28 8-bit possibilities for k(q), and let [image] be the mapping of the message by one round under the candidate k.

3. If [image] = k(q), then [image] = T(m) and

F( [image]) = FT(m) = TF(m) = T(m 16,m 17) = (m 17,?)

and hence the left-half of F( [image]) agrees with the right-half of F(m). The adversary obtains ciphertext F( [image]) corresponding to the chosen-text [image] if the right half of F(m) agrees with the left half of F( [image]) then [image] is considered a match for T(m), and [image] is accepted as a likely value of k(q). At most 28 = 256 values of [image] must be checked in order to obtain a match.

The procedure is applied for each q ∈ {0, 1}8, yielding a candidate for k using at most 28(28+1) = 65792 chosen-plaintexts.

The candidate for the key k may be incorrect. In step 3, [image] could be a ''match" without actually having the value T(m). However, if the cipher was designed to add confusion and diffusion, then we may assume that it is unlikely that more than one choice of [image] leads to a match at this stage.

We'd also like [image] = k(q) whenever [image] = T(m) at step 3. The only unknown part of the single-round calculation T(m) is the interchange condition k( [image]) on the output of the S 0, S 1 transformations. If the output of S 0 and S 1 agree on one of the bytes of m 1, then the corresponding bit of k( [image]) cannot be determined from an examination of T(m). Hence, in addition to the requirement that [image] = q, m should be chosen so that output of S 0 and S 1 differ in each byte of m 1. (The inability to choose such an m may indicate a system vulnerable to simpler attacks.)
< previous page | page_241 | next page >


page_242

< previous page | page_242 | next page >

Page 242

Toy example

Consider an NDS-like cipher with n = 4 and r = 3 rounds. Messages are 2n= 8 bits, and the key is a mapping k : {0, 1}2 → {0, 1}2. (Each of the 3 subkeys is k. The keyspace has size [image] Let S 0 be the identity, and let S 1 perform bitwise complement. The permutation writes the bits in reverse order. A schematic similar to Figure 10.3 has two pairs of S 0,S 1 boxes, each box accepting 1-bit of the n = 4 bits of mi.

Suppose the key k is defined by

k(00) = 10 k(10) = 11

k(01) = 00 k(11) = 10

and the message to be encrypted is m = (m 0,m 1) = (0111, 1100). The value of m 2 = m 0 ⊕ f(m 1) is calculated as

The other rounds are similar, giving

We illustrate the chosen-plaintext attack to recover k(q) for q = 10.

1. We wish to choose m = (m 0,m 1) so that [image] and so that the output of S 0 and S 1 on halves of m 1 differ. We may select m = (0111, 1100), obtaining F(m) = (1101, 0011).

2. Build a table of possible one-round encryptions [image] one entry for each guess [image] of k(q). Then F( [image]) corresponding to each chosen-plaintext [image] is obtained.

3. is considered a match whenever the left-half of F( [image]) agrees with the right-half of F(m) = (1101, 0011). Unfortunately, two matches have been obtained, corresponding to underlined entries in the table. At this stage, the possibilities for k(q) have been narrowed to [image] = 10 or [image] = 11.

The example illustrates that the proposed attack may fail to isolate the key value. Additional plaintexts could be tried; e.g., m = (0101, 1100) determines [image] = 11 as the correct value of k(10).
< previous page | page_242 | next page >


page_243

< previous page | page_243 | next page >

Page 243

Exercises

10.3.1 Given that c = (mr+1, mr) = (1111, 0100) is an output of the cipher in the toy example, find the corresponding message m.

10.3.2 Follow the procedure outlined in the toy NDS example to find k(00).

10.3.3 Define two functions f 0, f 1: {0, 1}4 → {0, 1}4 by

f 0(x 1, x 2, x 3, x 4) = (x 2 ⊕ x 4, 1, x 1 x 2, 1 ⊕ x 3)

f 1(x 1, x 2, x 3, x 4) = (1, x 1 ⊕ x 3, x 4, x 2).

Let F be the toy Feistel cipher defined as follows:

• n = 4 (so that messages are 2n = 8 bits long), and there are r = 2 rounds;

• the key of F is a pair (k 1,k 2) of binary digits;

• in round i ∈ {1,2}, the function [image] is used.

A message m is encrypted to ciphertext c under the key (0,1). If c = mr+1 mr = 10101011, then find m.

10.3.2 The Data Encryption Standard

DES is the result of a 1973 solicitation for encryption standards by the National Bureau of Standards (NBS, now NIST). Developed at IBM and modified jointly with the US National Security Agency (NSA), the cipher is also known as the Data Encryption Algorithm. DES is based on a 16-round Feistel cipher, acting on 64-bit input. The key schedule produces a 48-bit subkey ki for each of the rounds from a given 56-bit key k. There are eight fixed S-box mappings which are fundamental to the security of the system, shown in Figure 10.4.

The adoption of DES as a standard in 1977 was a milestone in the use of cryptography. DES would become the most widely-used symmetric-key scheme. The standard was controversial, in part because not all of the design principles were made public. The secrecy invited speculation that the scheme may contain a ''trap door" which permits the NSA to decrypt messages.

The size of the keyspace has been criticized heavily. The description of the scheme would be known to everybody; security rested entirely with the key. The standard specifies a keylength of 64 bits, of which 8 are parity, giving an effective keylength of 56 bits. An exhaustive search of the keyspace by a determined adversary with specialized hardware was a concern.

In 1977, Diffie and Hellman [28] proposed a DES exhaustive search machine at an estimated cost of US$20 million, searching the entire keyspace in a day. In 1993, Michael Wiener presented a detailed design for a machine which could search the entire keyspace in 7 hours for $1 million [93]; a 1997 version of the $1 million machine would perform the search in a little over an hour [94].
< previous page | page_243 | next page >


page_244

< previous page | page_244 | next page >

Page 244

Figure 10.4: Schematics for the Data Encryption Standard.

In January 1997, RSA Laboratories launched the ''DES Challenge," offering prizes for uncovering the DES key from three known-plaintext pairs. The DES-CHALL group claimed the prize in June 1997, using exhaustive search with a large-scale distributed network of computers connected by the internet. This was a volunteer effort, requiring 96 days with an estimated 70000 machines involved at some stage. Approximately 25% of the keyspace had been searched when the key was found. In 1998, a similar contest was solved in 40 days by distributed.net, searching 88% of the keyspace. Near the close of the contest, they recorded a peak rate of over 34 · 109 keys per second by nearly 1400 active teams.

In 1998, the Electronic Frontier Foundation (EFF) claimed a DES Challenge in 56 hours using a $0.2 million custom-built machine. In January 1999, distributed.net and the EFF combined forces, claiming a Challenge in a little over 22 hours, testing 245 · 109 keys per second when the key was found. There had been concern that US Government officials deliberately overestimated the costs of such a machine and underestimated the applicability of known-plaintext attacks, in order to protect other interests. As Diffie noted, by publishing the complete software and hardware details for their relatively inexpensive cracker [34], it is no longer "a question of whether DES keys can be extracted by exhaustive search; it is a question of how cheaply they can be extracted and for what purposes." The DES Challenge permitted known-plaintext attacks, but effective exhaustive search is possible if partial information concerning structure or content-type of the plaintext is known.

Multiple encryption (discussed below) can be used with DES to defeat exhaustive search, although there is a speed penalty (see also Exercise 10.3.5). DES

< previous page | page_244 | next page >


page_245

< previous page | page_245 | next page >

Page 245

is entrenched, however. As Whitfield Diffie remarked in the forward to the EFF cracker [34], ''The most convincing argument that DES is insecure would not out-weigh the vast investment in DES equipment that has accumulated throughout the world. People will continue using DES whatever its shortcomings, convincing themselves that it is adequate for their needs."

A possible DES replacement, the Advanced Encryption Standard (AES; see Section 10.4) was in the process of selecting an algorithm in 2000. Apparently, it will address the fundamental weakness of DES, the ability to effectively mount an exhaustive search of the keyspace. In [9], a group of well-known cryptographers recommended a 75-bit key to protect 1996 data and a 90-bit key for 20-year security, noting that "the cost of very strong encryption is not significantly greater than that of weak encryption."

Multiple encryption

Block ciphers such as DES may be applied in succession, in an attempt to obtain a cipher with a larger keyspace. Double encryption with a single cipher, for example, takes the form [image] for given keys k 1,k 2. Multiple encryption need not enhance security—in fact, it need not even enlarge the effective keyspace. If an encryption scheme is closed under composition in the sense that given k 1,k 2 there exists k 3 ∈ [image] so that [image], then double encryption offers no additional security. As an example, the simple substitution cipher over all permutations k of some alphabet (as in Example 10.2.2) has k 3 = k 2 º k 1.

For key k, DESk is a permutation on {0, 1}64. The keyspace defines at most 256 (out of a possible 264!) such permutations. This collection of permutations is not closed under composition, and multiple encryption is often used with the hope of defeating exhaustive search of the keyspace.

In the case of double-DES, an unsophisticated exhaustive search tries all (256)2 =2112 keys. However, double encryption is vulnerable to a meet-in-the-middle attack which reduces the effort to 257 DES operations at the cost of 256 storage. If at least one known-plaintext pair (m,c) is available, where [image] for unknown keys k 1,k 2, then the attack to recover the keys is:

1. Form a table of (i,DESi(m)) for all keys i, sorted by the second entry.

2. For each possible key j, determine if [image] (c) appears in the table; if so, then [image] (c) = DESi(m) for some i, and hence c = DESjDESi(m) and (i,j) is a candidate for (k 1, k 2). Additional known-plaintext pairs (if available) are used at this stage to discard spurious matches.

If at least two known-plaintext pairs are available, the attack is expected to determine the keys.

In practice, triple-DES is used, having a keyspace of size (256)3 = 2168. Given keys k 1,k 2,k 3, the encryption is [image], where Ek denotes DESk

< previous page | page_245 | next page >


page_246

< previous page | page_246 | next page >

Page 246

or [image]. The two-key triple-DES variant uses [image] and k 3 = k 1; choosing k 2 = k 1 gives DES. The meet-in-the-middle attack on triple-DES is no longer feasible, requiring on the order of 2112 DES operations. However, there are attacks on two-key triple-DES (requiring large numbers of either chosen-plaintext pairs or DES operations) showing it to be weaker than triple-DES [63].

Modes of operation

Block ciphers are often used in modes which process the plaintext in fragments (often the size of a block). Two such modes are considered in this section. DES will be used in the examples, but the ideas apply to any block cipher.

Let m = m 1 m 2... be a message, where each m i is a block (64 bits in DES). Electronic codebook (ECB) mode applies DES to each block, obtaining ci = DESk(m i). It has the advantage of simplicity, and bit errors in a ciphertext block affect decryption of that block only. A disadvantage is that identical messages (under the same key) map to the same ciphertext, possibly leaking information to an adversary.

Cipher-block chaining (CBC) chooses an initial block c 0 and encrypts as ci = DESk(m i ⊕ ci−1) for i ≥ 1. Decryption is performed by mi [image] (ci) ⊕ ci−1. The first block of plaintext can be chosen at random to prevent identical plaintexts from always giving the same output. Unlike ECB mode, there are chaining dependencies causing cj to depend on cj−1 (which depends on all preceding plaintext blocks).

In ECB mode, errors introduced (outside of the encryption process) into ciphertext c affect only the decryption of the damaged blocks. The chaining in CBC propagates an error in cj, affecting the recovery of both mj and mj+1. In fact, an adversary with the ability to modify cj can force bit errors into the recovery of mj+1 by changing the corresponding bits of cj (Exercise 10.3.4).

Applications to authentication

Symmetric-key systems can be used to provide authentication. We consider two specific examples, the first employing cipher-block chaining to provide message authentication, and the second providing identification (entity authentication) using a password scheme built from DES. The difference between entity and message authentication can be subtle, although the latter tends to involve an actual message rather than simply a claim of identity.

Message authentication Suppose Bob and Alice are exchanging messages (perhaps by email) over an unsecured channel. On receipt of a message purported

< previous page | page_246 | next page >


page_247

< previous page | page_247 | next page >

Page 247

to be from Alice, Bob should be able to convince himself that the message is genuine. At least two kinds of attacks are of concern: impersonation and forgery.

One possibility is to use a symmetric-key scheme such as DES. Alice and Bob share a secret key, and Alice encrypts the message m, sending c = E k(m). Bob decrypts c with the secret key, accepting the message if it ''makes sense." To prevent replay attacks—messages resent by an adversary—Alice could add time or sequence information to m before calculating c. If the cipher is such that it is difficult to create meaningful messages without knowledge of the key, then some protection from forgery is provided; however, some ciphers may allow selective changes to go undetected. For example, if a block cipher is applied in ECB mode, then the adversary could re-order, substitute, or delete ciphertext blocks. The one-time pad allows an adversary to force bit changes by changing the corresponding bits in the ciphertext.

A more robust scheme can be developed using cipher-block chaining. A block cipher E is selected and the message is written m = m 1...mt where each mi has bitlength matching the blocksize of the cipher (if E = DES, then the blocks are of length 64 bits). The block cipher is applied in CBC mode, giving c i = E k(m i ⊕ ci−1), 1 ≤ i ≤ t. Alice sends the message m along with ct, known as a message authentication code (MAC). Bob computes the CBC-MAC (as Alice did), and accepts the message as genuine if this value agrees with the received MAC. To defeat exhaustive search of the keyspace, a second key k′ may be selected and two-key triple-encryption performed on the last block (delivering [image] (ct) rather than ct as the MAC).

Identification Conventional password schemes rely on a secret shared between a user and the system; access to a system resource is granted when the user reveals the secret itself. We consider the traditional password mechanism used on many Unix computer systems: to gain access, a userid and password pair are presented to the system; access is granted if the system determines from its own stored information that the password corroborates the identity of the user.

A password of up to 8 characters is used in forming the key k for a modified DES encryption function. Each character provides 7 bits of the 56-bit key; zero bits are added if the password is less than 8 characters. An additional 12 bits (known as the salt) obtained from the system clock at the time of password creation is used to modify the expansion E in Figure 10.4, specifying one of 212 = 4096 variations. The system calculates [image] 1 ≤ i ≤ 25, where DES* denotes the modified DES scheme and m 0 is the 64-bit zero value. The 12-bit salt and 64-bit m 25 (known as the hashed password) are stored on the system, traditionally in /etc/passwd (a file). When subsequently presented with a userid and password, the system performs the same calculations, granting access if the result agrees with the stored value.

The calculation is known as the Unix crypt password algorithm. An adversary

< previous page | page_247 | next page >


page_248

< previous page | page_248 | next page >

Page 248

in possession of an entry from /etc/passwd can mount a known-plaintext attack, since the message input to the algorithm is known. Even if exhaustive search is infeasible, an attack using a dictionary of passwords which are known to be favored by users may be successful. The salt makes the dictionary attack more difficult, since 4096 variations of each password are possible. The salting procedure was also designed so that off-the-shelf DES hardware could not be used for password cracking.

Adversaries often obtain the /etc/passwd file itself (on many systems, all users could read this file), and perform various kinds of attacks, perhaps in the privacy of their own chambers. Recovery of some passwords is a significant threat, even by adversaries with relatively modest computing power.9 Systems that rely on this mechanism have countered by forcing users to choose better passwords (in an attempt to defeat dictionary attacks), and have moved the hashed passwords to a separate file which requires special privileges to read.

The password scheme is an example of weak authentication. The user receives no explicit corroboration of the identity of the system; if the channel is unsecured, then an adversary may be able to impersonate the system in addition to simple eavesdropping. We will return to authentication in Chapter 12.

Exercises

10.3.4 (Errors in DES ciphertexts) Suppose that t plaintext blocks m 1,...,mt are encrypted using DES producing ciphertexts c 1,...,ct, respectively.

(a) Suppose that one ciphertext, say cj, is transmitted incorrectly. Explain briefly the procedure to determine the number and location of blocks which will be decrypted incorrectly in ECB mode and CBC mode.

(b) Suppose CBC mode has been used and an adversary interchanges the ciphertext blocks c3 and c6. When the ciphertext is received, how many plaintext blocks will be in error?

(c) In CBC mode, show how an adversary forces specific bit errors in m j + 1 by adjusting cj.

10.3.5 This problem concerns proposals to protect DES from exhaustive key search. The key is k = (k 1,k 2) where k 1 ∈ {0, 1}56 and k 2 ∈ {0, 1}64. Let m ∈ {0, 1}64 be the plaintext message.

(a) Show that encryption defined by E k(m) = DESk1(m) ⊕ k 2 does not increase the work needed to break the encryption scheme using exhaustive search. That is, show how to break this scheme using on the order of 256 DES operations. You may assume that you have a moderate number of pairs (m i,c i = E k(m i)).

9Feldmeier and Karn [32] report that their 1989 dictionary attacks typically recovered over 30% of the passwords on a given system. The authors present a fast crypt algorithm, used in password-cracking attacks where the hashed passwords have been obtained.
< previous page | page_248 | next page >


page_249

< previous page | page_249 | next page >

Page 249

(b) Does encryption defined by E k(m) = DESk1(m ⊕ k 2) make exhaustive search less feasible?

A ''DESX" extension suggested by Rivest is discussed in the proceedings of CRYPTO '96 [49]. For key k = (k 1,k 2,k 3), the encryption function is Ek(m) = k 3 ⊕ DES k1 (m ⊕ k 2). The inexpensive XOR pre- and post-processing may be used with existing DES machinery.

10.3.6 Suppose that Eve has obtained three pairs (m1, c1), (m2, c2), and (m 3,c 3) which were encrypted by Alice using triple-DES with E(m) = DESk3 DESk2 DESk1(m). Devise a meet-in-the-middle attack that recovers Alice's key (k 1,k 2,k 3) in roughly 2112 operations.

10.3.7 (DES complementation property) Let [image] denote the bitwise complement of a bit string m. If c = DESk(m), then an examination of the description of DES establishes that [image]. Does it appear that this property can be used to improve the running time of exhaustive key search under a known-plaintext attack? Under a chosen-plaintext attack?

10.3.8 The CBC-MAC described provides data integrity but not confidentiality. Consider the following proposal to add secrecy. The MAC is appended to message m = m 1...mt, giving m′ = mm t+1. Then m′ itself is encrypted in CBC mode (using the same initial c 0 and key), giving c 1...c t+1, where c i = E k(mi ⊕ c i−1), 1 ≤ i ≤ t + 1; the first t of these calculations are identical to those performed in obtaining the MAC.

The method appears attractive, obtaining the ciphertext directly from the calculations for the MAC. Show that the scheme results in a final ciphertext block ct+1 = Ek(mt+1 ⊕ ct) which is independent of the plaintext and the ciphertext. Explain why this means that the added encryption has compromised the authentication, and illustrate how an adversary may exploit this fundamental flaw.

10.4 Notes

The opening sentence of this chapter is from Rivest [71], an excellent introduction to cryptography. Simmons [81] contains contributions from several authors, including The First Ten Years of Public Key Cryptography by Diffie [26]. The Handbook of Applied Cryptography by Menezes, van Oorshot, and Vanstone [63] is recommended for its well-organized and comprehensive coverage.

Kahn's The Codebreakers is often cited for its complete (and lively) nontechnical history of cryptology up to 1967; a few notes on more recent developments appear in a revised 1996 edition [48]. Garfinkel [37] discusses some of the political, legal, privacy, and policy questions concerning cryptography, along with the history of the Pretty Good Privacy (PGP) application. The textbooks of Stinson [86] and Koblitz [50] are natural choices to extend the material introduced here.
< previous page | page_249 | next page >


page_25

< previous page | page_25 | next page >

Page 25

If d is odd, say d = 2t+1, then

so d(v,v+u) > d(w,v+u). And if d is even, say d = 2t, then

In either case, d(v,v+u) ≥ d(w,v+u), so v+u is not closer to v than to the codeword w. Thus C does not correct the error pattern u.

In view of this theorem it is clear that any code of distance d is a [image] error-correcting code.

Example 1.12.10 The code C = {000, 111} has distance d = 3. Since [image] Theorem 1.12.9 ensures that C corrects all error patterns of weight 0 or 1. As we observed in Example 1.12.1, C does correct error patterns 000, 100, 010, and 001. The error pattern 110 has weight [image] and we saw that C does not correct 110.

Theorem 1.12.9 does not prevent a code C of distance d from correcting error patterns of weight greater than [image].

Example 1.12.11 Let C = {001, 101}. Then d = 1. The error pattern u = 011 has weight 2, which is greater than [image]. As the following piece of the IMLD table shows, C does correct u = 011.

w | 001+w | 101+w | v
010 | 011* | 111 | 001
110 | 111 | 011* | 101

Exercises

1.12.12 For each of the following codes C: (i) determine the error patterns that C will correct (the IMLD tables for these codes were constructed in Exercise 1.9.7), and (ii) find the error patterns that Theorem 1.12.9 guarantees that C corrects.

(a) C = {101, 111, 011}

(b) C = {000, 001, 010, 011}

(c) C = {0000, 0001, 1110}

(d) C = {0000, 1001, 0110, 1111}

(e) C = {00000, 11111}

(f) C = {00000, 11100, 00111, 11011}

(g) C = {00000, 11110, 01111, 10001}

(h) C = {000000, 101010, 010101, 111111}

1.12.13 Use the technique described in Example 1.12.11 to decide whether or not the following error patterns are corrected by the accompanying code.
< previous page | page_25 | next page >


page_250

< previous page | page_250 | next page >

Page 250

Extensive material from the VENONA project at the National Security Agency is available via http://www.nsa.gov. The historical introduction begins with the following paragraph.

On 1 February 1943, the U.S. Army's Signal Intelligence Service, a forerunner of the National Security Agency, began a small, very secret program, later codenamed VENONA. The object of the VENONA program was to examine and possibly exploit, encrypted Soviet diplomatic communications. These messages had been accumulated by the Signal Intelligence Service (later renamed the U.S. Army Signal Security Agency and commonly called ''Arlington Hall" after the Virginia location of its headquarters) since 1939 but had not been studied previously. Miss Gene Grabeel, a young Signal Intelligence Service employee, who had been a school teacher only weeks earlier, started the project.

The description of the New Data Seal is from Beker and Piper [3]. Details of DES may be found in [63, 86, 3]. Early commentary on the "DES controversy" appears in a series of IEEE Spectrum articles beginning with [87]. Some design criteria for the S-boxes may be found in Coppersmith [23], Stinson [86], and Schneier [76]. RSA's DES Challenges are described on http://www.rsa.com. The non-profit Electronic Frontier Foundation maintains a web page on various civil liberties issues at http://www.eff.org; on-line versions of Cracking DES [34] are available.

Use of DES CBC has been standardized by the International Organization for Standards (ISO 9797) and the American National Standards Institute (ANSI X9.9) for authentication purposes [63]. ANSI X9.9 is widely used by banks in financial applications. Triple-DES was approved by ANSI in November 1998 as a standard (ANSI X9.52). In 1999, the National Institute of Standards and Technology (NIST) was in the process of considering triple-DES for a US Government Federal Information Processing Standard (FIPS 46-3; see http://csrc.nist.gov/cryptval). The draft announcement contained:

Additionally, knowing that the DES' security life was nearing an end, NIST has been working with industry and the cryptographic community to develop an Advanced Encryption Standard (AES) for the 21st century. On January 2, 1997, NIST announced the initiation of an effort to develop the AES (62 FR 93). It is intended that the AES will specify an unclassified, publicly disclosed encryption algorithm capable of protecting sensitive government information well into the next century. Unfortunately, since it takes a substantial amount of time to gain confidence in a new encryption algorithm, the AES is not expected to be a fully developed FIPS for some time to come. Information on NIST's multi-year effort to develop the AES can be obtained at http://www.nist.gov/aes.

Recently claims have been made of a special-purpose hardware based attack on the DES. In light of this most recent attack, NIST can no longer support

< previous page | page_250 | next page >


page_251

< previous page | page_251 | next page >

Page 251

the use of the DES for many applications. As with other security tools, encryption must balance cost against risk. The recent brute force exhaustion attack by a "cracking machine" costing $250000 took 56 hours to crack a single message. With this special-purpose technology, the average time of cracking per message would be twice that, since only a quarter of all keys were tested. In some cases this kind of attack may not pose an immediate or significant threat—for example where short-term protection of perishable information is desired. However, advances in technology are likely to further reduce the average cracking time. Therefore, NIST recommends the following:

• For existing systems, develop a prudent transition strategy to move to Triple DES. This strategy should match the strength of the protective measures against the associated risk. Critical systems should receive priority.

• When building new systems, use Triple DES to protect sensitive, unclassified data.

These recommendations are reflected in the proposed draft FIPS 46-3...by recognizing Triple DES, as described in ANSI X9.52, as a FIPS approved algorithm.
< previous page | page_251 | next page >


page_253

< previous page | page_253 | next page >

Page 253

Chapter 11
Topics in Algebra and Number Theory

Public-key cryptography relies for security on the (supposed) intractability of various number-theoretic problems. The integer factorization problem (FACTOR) and the discrete logarithm problem (DLP) are well-known examples:

FACTOR: Find the prime factorization of a given positive integer n.

DLP: Given prime p, generator α of [image] and αx mod p, find x.

To be slightly more precise, public-key schemes require two (related) problems, one which is computationally feasible, and one which is intractable. In the case of integer factorization, for example, it is relatively easy to find the product n = pq; efficient recovery of p and q given n in the general case remains an open problem.

Four topics are considered in the following sections: quadratic residues, primality testing, integer factorization, and discrete logarithms. Clearly, the coverage will not be exhaustive, and the goal is to consider a respectable sampling of material from algebra and number theory (and algorithmic number theory) fundamental to public-key schemes.

11.1 Algorithms, complexity, and modular arithmetic

Several algorithms involving arithmetic operations on integers are presented in this section. One measure of efficiency is a count of the number of operations involving single-bit arithmetic. For example, if x and y are k-bit binary numbers, then x+y may be done in at most k bit operations, and the usual approach to calculating xy requires at most k 2 bit operations (there are at most k − 1 additions, each requiring at most k bit operations). These are ''worst-case" counts, over all integers with at most k bits. We can regard the number of bit operations as a measure of the running time, in terms of the "size" k of the input, required by an algorithm.
< previous page | page_253 | next page >


page_256

< previous page | page_256 | next page >

Page 256

If (a,b) = 1, then a and b are said to be relatively prime. For n ≥ 1, the number of integers in the interval [1,n] which are relatively prime to n is denoted φ(n), known as the Euler phi function. For example, φ(6) = 2 and φ(pi) = pi−1 (p−1) if p is prime. The phi function is multiplicative in the sense that φ(ab) = φ(a)φ(b) if (a,b) = 1; in particular, φ(pq) = (p − 1) (q − 1) for primes p ≠ q.

The integers modulo n

Let n be a positive integer. Then a is congruent to b modulo n, written a ≡ b (mod n), if n|a−b. As examples, 14 ≡ 9 (mod 5), –11 ≡ 3 (mod 7), and –1 ≡ n – 1 (mod n). Congruence is an equivalence relation on [image]; that is, congruence is reflexive (a ≡ a (mod n)), symmetric (if a ≡ b (mod n) then b ≡ a (mod n)), and transitive (if a ≡ b (mod n) and b ≡ c (mod n) then a ≡ c (mod n)). If a = qn + r with 0 ≤ r < n, then a ≡ r (mod n). Hence every integer a is congruent to a unique integer in [0,n − 1], namely r = a mod n. An equivalence (or residue) class [a] consists of all integers congruent to a modulo n; the set of equivalence classes modulo n is denoted [image].

It can be shown that the operations of addition and multiplication on [image] given by [a] + [b] = [a + b] and [a][b] = [ab], respectively, are well-defined; that is, if a ≡ a′ (mod n) and b ≡ b′ (mod n) then a + b ≡ a′ + b′ (mod n) and ab ≡ a′b′ (mod n). Under these definitions of addition and multiplication, ( [image], +,·) is a ring (see Exercise 11.1.12). The class notation is commonly omitted, and the set of equivalence classes is written [image] = {0,...,n − 1} with the understanding that an integer a is identified with its equivalence class [a].

Given a ∈ [image], if there exists x ∈ [image] such that ax ≡ 1 (mod n), then a is said to be invertible and its inverse x is denoted a−1. For example, in [image] 2−1 = 5 since 2 · 5 ≡ 1 (mod 9); the elements in {0,3,6} do not have inverses. If a ∈ [image] is invertible, then ax ≡ (mod n) and n|ax−1 so that ax − ny = 1 for some integer y, and (a,n) must be 1. Conversely, if (a,n) = 1, then x and y exist by the extended Euclidean algorithm. Hence, a ∈ [image] is invertible if and only if (a,n) = 1, in which case the inverse may be found by the extended Euclidean algorithm.

Example 11.1.3 Let a = 7 and n = 9. The first column uses the Euclidean algorithm to show that (7,9) = 1, and hence a is invertible in [image]. The second column finds integers x and y so that xa + yn = (a,n).

Hence, 7−1 = 4, which can be verified by 7 · 4 ≡ 28 ≡ 1 (mod 9).
< previous page | page_256 | next page >


page_257

< previous page | page_257 | next page >

Page 257

The set [image] = {a ∈ [image] | (a, n) = 1} of invertible elements of [image] forms a group under multiplication, containing φ(n) elements. For example [image] = {1, 5, 7, 11} and [image] = {1,..., p – 1} if p is prime. For prime p and a ∈ [image], Fermat's (little) theorem says ap−1 ≡ 1 (mod p); a generalization due to Euler shows a φ(n) ≡ 1 (mod n) for integer n ≥ 2 and a ∈ [image]. In particular, we may define the order of a ∈ [image], written ord(a), as the least positive integer t such that at ≡ 1 (mod n). Exercise 11.1.21 establishes some elementary properties concerning [image]; in particular, ord(a) | φ(n) for a ∈ [image]. If a ∈ [image] has ord (a) = | [image] | = φ(n), then a is said to be a generator of [image]; in this case, [image] = {ai | 0 ≤ i < φ(n)}. It is known that [image] for p prime has a generator; for example, it is easily verified that 2 is a generator of [image].

Example 11.1.4 Consider [image] = {1,2,4,7,8,11,13,14}, having φ(15) = φ(3)φ(5) = (3−1)(5−1) = 8 elements. The orders of the elements are:

As promised, the order of every element divides φ(n) = 8. Note that [image] does not have a generator (since no element has order 8).

Theorem 11.1.5 (Chinese remainder) If the integers nq,...,nk are pairwise relatively prime, then the system of congruences

has a unique solution modulo n = n 1...nk.

Consider the case k = 2 in the Chinese remainder theorem. Since (n 1,n 2) = 1, there exist integers s and t so that sn 1+tn 2 = 1. The verification that x = (sn 1 a 2 + tn 2 a 1) mod n is the solution to the system in Exercise 11.1.18. As a specific example, consider

x ≡ (mod 7),

x ≡ (mod 13).

The extended Euclidean algorithm gives (7,13) = 1 = 2·7 − 1·13, and then x = 2·7·6 − 1·13·3 = 45 is the unique solution (modulo n = 7·13 = 91) to the pair of congruences. The generalization is Gauss' algorithm.

Algorithm 11.1.6 (Gauss) The solution x to the system in the Chinese remainder theorem may be computed as [image], where Ni = n/ni and [image]. The computations can be performed in [image] bit operations.
< previous page | page_257 | next page >


page_258

< previous page | page_258 | next page >

Page 258

Table 11.1: Bit complexity of basic operations in [image].

Efficient calculation of ak mod n is essential in several cryptographic schemes. The calculation can be done naively by computing ak and dividing by n or by computing ai mod n, 1 ≤ i ≤ k, by repeated multiplication; unfortunately, these calculations are inefficient in terms of storage or number of multiplications required. The method of repeated square-and-multiply provides an efficient calculation. The exponent is written in binary as [image], where t = [image] logk [image], and then

Note that [image], and hence the value ak mod n can be calculated with t modular squarings and at most t modular multiplications. We present an algorithm for modular exponentiation which proceeds by calculating the partial products; other square-and-multiply algorithms may be found in [63]. Bit complexities for the basic operations in [image] appear in Table 11.1.

Algorithm 11.1.7 (exponentiation by square-and-multiply) Input: 0 ≠ a ∈ [image], and integer 0 ≤ k < n with binary representation [image]. Output: ak mod n.

1. Set A ← a and b ← 1.

2. For i from 0 to t do the following:

2.1 If i > 0 then set A ← A 2 mod n.

2.2 If ki = 1 then set b ← bA mod n.

3. Return (b).

Example 11.1.8 The algorithm is applied to 326 mod 35, i.e., a = 3, k = 26 = [image] = 110102, and n = 35. The arithmetic can be collected in tabular form as follows:
< previous page | page_258 | next page >


page_259

< previous page | page_259 | next page >

Page 259

and hence 326 mod 35 = 9. In this particular example, we may quickly verify by reducing the exponent modulo φ(n) = φ(5·7) = 24, obtaining 3k mod n = 32.

Exercises

11.1.9 Verify the following properties of divisibility: (a)a|a; (b)if a|b and b|c, then a|c; (c)if a|b and b|a, then a = ±b; (d) if c|a and c|b, then c|ax + by for all x,y.

11.1.10 Define the least common multiple lcm(a,b) of positive integers a and b by lcm(a,b) = ab/(a,b). Show that lcm(a,b) | c whenever a|c and b|c.

11.1.11 This question concerns the Euclidean algorithm 11.1.1.

(a) Show that the remainders satisfy r i + 2 < ri/2.

(b) Show that the algorithm returns (a,b).

11.1.12 This exercise examines elementary properties of [image].

(a) Show that the operations of addition and multiplication on [image] are well-defined in the sense described in the text.

(b) The set [image] together with the defined addition and multiplication forms a ring, denoted ( [image], +,·). The ring properties follow directly from corresponding properties in [image]. Verify that multiplication distributes over addition; i.e., ([a]+[b])[c] = [a][c]+[b][c].

11.1.13 Use the Euclidean algorithm to find d = (105,180). Then find integers x and y such that 105x+180y = d.

11.1.14 Use the square-and-multiply algorithm to compute 47332 mod 576.

11.1.15 The following statements are false in general. Give a counterexample to each statement.

(a) Let a,b,n ∈ [image] If n|ab then n|a or n|b.

(b) Let p be a positive integer and a ∈ [image] If (a,p) then a p–1 ≡ 1 (mod p).

(c) If a, b, c ∈ [image], then (ab,c) = (a,c)(b,c).

11.1.16 Find the order of each element of [image]. Identify the generators of [image].

11.1.17 Let a ∈ [image]. Prove that the elements a i mod n 0≤i<ord(a), are pairwise distinct.

11.1.18 Verify the claim made in the paragraph following the Chinese remainder theorem 11.1.5 that x = (sn 1 a 2+tn 2 a 1) mod n (where sn 1+tn 2 =1) is the solution to the system of two congruences.

11.1.19 Determine if the pair of congruences

x ≡ 15 (mod 70)

x ≡ 104 (mod 151)

has a solution. If so, use the extended Euclidean algorithm to find all solutions.
< previous page | page_259 | next page >


page_26

< previous page | page_26 | next page >

Page 26

(a) C = {000000, 100101, 010110, 001111, 110011, 101010, 011001, 111100}

(i) u = 001000 (ii) u = 000010 (iii) u = 100100

(b) C = {1001011, 0110101, 1110010, 1111111}

(i) u = 0100000 (ii) u = 0101000 (iii) u = 1100000

1.12.14 For each code in Exercise 1.12.12, find an error pattern of weight [image] +1 that C does not correct.

1.12.15 Let C be the code consisting of all words of length 4 having even weight. Determine the error patterns that C will correct.

1.12.16 Let u 1 and u 2 be error patterns of length n, and assume that u 1 and u 2 agree at least in the positions where a 1 occurs in u 1. Prove that if a code C will correct u 2, then C will also correct u 1.

As we have observed, error patterns of small weight are more likely to occur than error patterns of large weight(Theorem 1.6.3). Therefore, in designing codes, we should concentrate on being able to correct, or at least to detect, error patterns of small weight.
< previous page | page_26 | next page >


page_260

< previous page | page_260 | next page >

Page 260

11.1.20 Prove that the congruence ax ≡ b (mod n) has a solution x if and only if (a,n)|b, in which case there are exactly (a, n) solutions.

11.1.21 Let a∈ [image]. Prove the following claims:

11.1.22 Let p be prime and let a ≥ 1 be an integer. Prove that the number of solutions in [image] to xa ≡ 1 (mod p) is (a, p − 1).

11.1.23 If p is prime, then every nonzero element of [image] has an inverse and [image] is a field. Fermat's little theorem says that [image] An elementary proof may be given as follows.

(a) Let T = {a, 2a, ...,(p−1)a} [image] Show that the elements of T are nonzero and pairwise distinct.

(b) Part (a) shows that [image]. Hence 1·2...(p – 1) ≡ a(2a)...(p – 1)a (mod p). Use division to complete the proof.

11.2 Quadratic residues

Let a [image]. Then a is a quadratic residue modulo n if there exists x [image] with x 2 ≡ a (mod n). In this case, x is a square root a modulo n. Otherwise, a is a quadratic nonresidue modulo n.

The set of quadratic residues modulo n is denoted by Qn; the nonresidues are [image]. Note that [image]. There are elements a ∈ [image] for which x 2 ≡ a (mod n) has a solution, but a is not a quadratic residue; for example, [image] has 22 ≡ 4 (mod 6) but 4 ∉ Q 6 (since [image]).

Theorem 11.2.1 Let p > 2 be prime and let α be a generator of [image]. Then a ∈ [image]. is a quadratic residue modulo p if and only if a ≡ α2i (mod p) for some i ∈ [image].

Proof: If a = α2i mod p for some i, then x = αi satisfies x 2 ≡ a (mod p) and a is a quadratic residue. Conversely, if a ∈ Qp, then x 2 ≡ a (mod p) has a solution x ≡ αi (mod p) for some i, and α2i ≡ a (mod p).

Corollary 11.2.2 Let p ≥ 2 be prime and let α be a generator of [image] .
< previous page | page_260 | next page >


page_261

< previous page | page_261 | next page >

Page 261

Proof: The first two are immediate consequences of the theorem. The third follows since there are at least two solutions ±x by definition of a quadratic residue, and statement 2 of this corollary shows that there cannot be more than two solutions. For the fourth, note that α(p−1)/2 is a solution of x 2 ≡ 1 (mod p) since the order of α is p − 1. Necessarily, [image] (mod p), and only the congruence in 4 is compatible with the order of α.

For a given [image], an exhaustive search through the elements [image] [image] will, of course, determine if a is a quadratic residue. This is not efficient, requiring O(p) modular multiplications in the worst case. The corollary can be used to develop a more efficient criterion. Define the Legendre symbol [image] for prime p > 2 and integer a by

Theorem 11.2.3 (Euler's criterion) Let p > 2 be prime and a ∈ Z. Then [image] ≡ [image].

Proof: We consider the case that a ∈ Qp; the cases [image] and p | a are left as Exercise 11.2.15. Let α be a generator of [image]. Then there exists i such that α2i ≡ a (mod p) by Theorem 11.2.1. By definition, [image] = 1, and a(p − 1)/2 ≡

Euler's criterion suggests using [image] to determine if a given a is a quadratic residue modulo the prime p. This can be done efficiently [image] bit operations) using a square-and-multiply approach. However, unlike exhaustive search, it will not exhibit a square root in the case a ∈ Qp.

Example 11.2.4 Euler's criterion can be used to determine if 3 is in Qp for p = 23. To calculate 3(p−1)/2 mod p, write

Following Algorithm 11.1.7, the table for the arithmetic is

i | 0 | 1 | 2 | 3
ki | 1 | 1 | 0 | 1
A | 3 | 32=9 | 92 mod p = 12 | 122 mod p = 6
b | 3 | 3.9 mod p = 4 | 4 | 4.6 mod p = 1

and [image] mod p = 1. This shows that 3 ∈ Q 23, and hence there is an x with x 2 ≡ 3 (mod 23). Later, we will have an efficient algorithm for exhibiting such an x. In this toy example, it is easily verified that 72 ≡ 3 (mod 23).
< previous page | page_261 | next page >


page_262

< previous page | page_262 | next page >

Page 262

Properties of the Legendre symbol can be used to calculate [image] more efficiently than the method suggested by the example. We shall return to this question after developing an extension of the Legendre symbol.

Let n ≥ 3 be an odd integer with prime factorization [image] and let a ∈ Z. The Jacobi symbol [image]. is defined in terms of the Legendre symbol by

Properties of the Jacobi symbol Let m, n ≥ 3 be odd integers, and let a, b ∈ Z.

Applying these properties to the calculation of (3/23) in Example 11.2.4, we obtain

Since 23 is prime, this establishes that 3 ∈ Q 23, as seen earlier using Euler's criterion.

The definition of the Jacobi symbol [image] suggests that n must be factored, which is believed to be a hard problem. However, the properties can be used to evaluate the symbol without factoring.

Example 11.2.5 Properties of the Jacobi symbol may be used to evaluate (28/55) as follows:

Although the Jacobi symbol is 1, we cannot conclude that 28 is a quadratic residue modulo 55 (in fact, it isn't). Note that 28(55−1)/2 ≡ 52 (mod 55), illustrating that

< previous page | page_262 | next page >


page_263

< previous page | page_263 | next page >

Page 263

Euler's criterion need not hold for composites. Finally, recall that a quadratic residue modulo a prime has exactly two square roots. Here, n = 55 is not prime, and, for example, 1 has square roots 1 and 55 − 1 and also 21 and 55 − 21. (Roots are discussed in Section 11.4.)

As the example shows, the condition [image] = 1 does not distinguish quadratic residues from nonresidues in the case of composite n. However, if a ∈ Qn, then there exists x ∈ [image] such that x 2 ≡ a (mod n) and hence [image]. Stated in the contrapositive, this says that [image] = −1 implies that a is a quadratic nonresidue. For odd integers n ≥ 3 we define

The elements of [image] = Jn \ Qn are called pseudosquares modulo n. Note that Qn [image] Jn = Qn in the case that n is prime.

Example 11.2.6 The quadratic residues and pseudosquares for n = 15 are obtained. Note that (a/15) = (a/3)(a/5) and

The Jacobi symbols [image] are calculated in the following table:

Hence, J 15 = {1, 2, 4, 8}. It can be verified that Q 15 = {1, 4}, and then the pseudosquares are given by [image].

Determining if a given a ∈ Jn is a quadratic residue or pseudosquare modulo n is known as the quadratic residuosity problem (QRP). Consider the special case n = pq for distinct odd primes p and q. It is shown in the exercises that a ∈ Jpq is a quadratic residue if and only if a ∈ Qp, and that [image]. Applied to Example 11.2.6, [image] if and only if a ∈ {1, 4}; necessarily, Q 15 = {1, 4} and |Q 15| = 2 = (3 − 1)(5 − 1)/4.

Exercises

11.2.7 Find Q 30 and [image].

11.2.8 Evaluate the Jacobi symbols (156/235) and (1833/587). Is 156 ∈ Q 235?

11.2.9 Find the quadratic residues and pseudosquares for n = 21.
< previous page | page_263 | next page >


page_264

< previous page | page_264 | next page >

Page 264

11.2.10 A careless colleague remarks ''36 is a quadratic residue modulo n for any n > 36 since 62 = 36." Correct the statement, and then determine if 36 is in Q 745.

11.2.11 Assume that the following properties of Legendre symbol are known:

• (−1/p) = (−1)(p−1)/2 and (2/p) = (−1)(p 2−1)/8 for odd primes p.

• If p and q are odd primes, then [image].

Let n ≥ 3 be an odd integer. Prove the following facts about the Jacobi symbol.

(a) Show that if n 1 and n 2 are odd integers, then [image] (mod 2). Hence prove that [image].

(b) Show that if n 1 and n 2 are odd integers, then [image]. Hence prove that [image].

(c) Prove that if a ≥ 3 is an odd integer, then

11.2.12 Let p be an odd prime. Prove that −3 ∈ Qp if and only if p ≡ 1 (mod 3).

11.2.13 Prove that a quadratic residue can never be a generator of [image].

11.2.14 Let n = pq for distinct odd primes p and q.

(a) Show that a ∈ Qn iff [image].

(b) Show that [image] Suggestion: show that the map f : Qn → Qp × Qq defined by f(a) = (a mod p, a mod q) is a bijection.

11.2.15 Complete the proof of Euler's criterion (Theorem 11.2.3).

11.3 Primality testing

The generation of (large) prime numbers is a basic requirement in several publickey schemes. Determining if a given integer n > 2 is prime or composite is a sub-problem. Trial division by all integers (or all primes) between 2 and [image] will, of course, determine if n is prime. It also has the pleasant feature of exhibiting a nontrivial factor in the case that n is composite; unfortunately, trial division requires [image] divisions.

In this section we consider two probabilistic primality tests, Solovay-Strassen and Miller-Rabin. They are probabilistic in the sense that the test concludes either "composite" (in which case n is certainly composite) or "probably prime" (in which case n may actually be composite). For this reason, the tests are more accurately called tests for compositeness.

Solovay-Strassen is built on Euler's criterion (Theorem 11.2.3): if n is prime, then [image]. (mod n). This motivates the following definition.

Definition 11.3.1 Let n be an odd composite integer and 1 ≤ a < n. If (a, n) ≠ 1 [image] (mod n) then a is an Euler witness (to compositeness) for n.
< previous page | page_264 | next page >


page_265

< previous page | page_265 | next page >

Page 265

Theorem 11.3.2 Let n be an odd composite integer and let a ∈ [image] be an Euler witness for n. Then at least half of all elements in [image] are Euler witnesses for n.

Proof: The set [image] (mod n)} of non-witnesses is closed under the product in the finite group [image], and hence G is a subgroup. By Lagrange's theorem, |G| is a divisor of [image]. Since a ∈ [image] \ G, it follows that [image].

As an alternative argument, verify directly that ab is an Euler witness for n whenever b ∈ [image] is not a witness. Necessarily, the number of non-witnesses in [image] is at most [image].

Theorem 11.3.3 Let n be an odd composite integer. Then there exists an Euler witness for n in [image] .

Proof: First, suppose that n is not square-free, i.e., p 2 | n for some prime p. Let a = 1+n/p. Then

and, in particular, a ∈ [image]. Also

Hence ord(a) = p since [image] (mod n). Since [image] it follows that [image] 1 (mod n), and a is an Euler witness.

Now consider the case that n is the product of distinct primes. Let p be a prime factor of n, and let b be a quadratic nonresidue modulo p. By the Chinese remainder theorem, there exists an a satisfying

By properties of the Jacobi symbol,

(in particular, a ∈ [image]. The definition of a implies that a(n−1)/2 ≡ 1 (mod n / p), and hence [image] (mod n/p). It follows that [image], and a is an Euler witness.

< previous page | page_265 | next page >


page_266

< previous page | page_266 | next page >

Page 266

Roughly speaking, the Solovay-Strassen test looks for Euler witnesses for a given n, and, if none are found, concludes that n is probably prime. The number of Euler witnesses for a composite n is used to give an error bound.

Algorithm 11.3.4 (Solovay-Strassen) Input: an odd integer n > 2 and security parameter t ≥ 1. Output: an answer of ''composite" or "probably prime."

1. Do the following (at most) t times:

1.1 Select a random integer a, 1 < a < n.

1.2 If (a,n) ≠ 1, then return ("composite").

1.3 If [image] (mod n), then return ("composite").

2. Return("probably prime").

The running time for the inner loop is [image] bit operations. The algorithm has been written for clarity; however, note that step 1.2 may be replaced by a comparison on a(n−1)/2 mod n, terminating with "composite" if the value is not 1 or n − 1.

If the test outputs "composite," then a certificate is provided which allows efficient verification that n is indeed composite. For Solovay-Strassen, the certificate is an Euler witness for n, and the verification algorithm checks that (a,n) ≠ 1 or that [image] (mod n). The probability that the test outputs "probably prime" when n is composite is at most 2−t.

Tests such as Solovay-Strassen (and Miller-Rabin, discussed in the exercises) can be used to generate large random probable primes: choose random n of the desired size until the test declares n to be "probably prime." In practice, n may first be subjected to divisibility tests by small primes, and there may be additional constraints depending on the application.

Exercises

11.3.5 Let n be an odd composite integer and let 1 ≤ a < n. If [image] (mod n), then a is said to be a Fermat witness (to compositeness) for n. Prove that if a is a Fermat witness for n, then a is also an Euler witness for n.

11.3.6 Perform the Solovay-Strassen test on n = 91. Choose a = 74 as your first "random" a. Unless you are rather unlucky, your second random a should determine that n is composite.

11.3.7 (Miller-Rabin probabilistic primality test) Let n be an odd integer, and let n − 1 = 2s r where r is odd. Let a ∈

Fact. If n is prime, then either [image] (mod n) for some j, 0 ≤ j < s.

Definition. Suppose that n is composite. If [image] (mod n) and if [image] (mod n) for all j, 0 ≤ j < s, then a is called a strong witness for n.
< previous page | page_266 | next page >


page_267

< previous page | page_267 | next page >

Page 267

Fact. If n ≠ 9 is an odd composite integer, then at least three-quarter of all a ∈ [image] are strong witnesses for n.

(a) Using the above concepts and facts, devise a test for determining the primality of an integer n.

(b) What is the running time (number of bit operations) of your test?

(c) Discuss the correctness of the results provided by this primality test.

11.4 Factoring and square roots

The problem of resolving a composite integer n into its prime factors is of more than academic interest. Some widely-used cryptographic schemes (such as RSA, discussed in Chapter 12) depend completely on the intractibility of factoring carefully chosen composite n. Factoring such an n can bring a measure of fame: the New York Times devoted front page coverage in 1988 to the factoring of a 100-digit integer by the quadratic sieve (discussed in Section 11.4.2) with a distributed network of 400 computers.1

No efficient method for factoring general n is known. A special-purpose algorithm can be effective when n is of certain form; for example, simple trial division can be used to find small factors, and Pollard's rho (discussed below) may efficiently find relatively small factors of n. The factoring challenge discussed in the New York Times article concerned n = pq for primes p and q of 41 and 60 decimal digits, respectively, specifically chosen to defeat certain special-purpose algorithms. For such cases, a general-purpose algorithm from the random squares family has been the method of choice.

For some material which follows, there is an implicit assumption that n has at least two distinct prime divisors. An integer which is a perfect power (i.e., of the form xk for some integers x and k > 1) is easily detected and, moreover, nontrivial factors are easily obtained (Exercise 11.4.9).

11.4.1 Pollard's rho

Consider the following problem. Let X be a finite set, let f : X → X, and let x 0 ∈ X. We can define a sequence (xi) ⊆ X as follows: x 1 = f(x 0), x 2 = f(x 1), and, in general, x i = f(x i−1) for each positive integer i. Since X is finite, this sequence must eventually start repeating. In particular, for some i < j, we must have x i = x j. The problem is to find such a pair (i,j).

1 Malcolm W. Browne, ''A Most Ferocious Math Problem Tamed," 12 October 1988. Apparently, the title was insufficiently dramatic for the factoring enthusiast among the NYT editorial staff who inserted the heading "World's Fiercest Math Problem Is Tamed by Hundreds of Computers."
< previous page | page_267 | next page >


page_268

< previous page | page_268 | next page >
Page 268 This page intentionally left blank.
< previous page | page_268 | next page >


page_269

< previous page | page_269 | next page >

Page 269

Here are the details, for our polynomial f with integer coefficients, and any x 0 ∈ [image]. Begin with x = x 0 and y = x 0. Repeat the two calculations

x ← f(x) mod n and y ← f(f(y)) mod n

until d = (x − y, n) > 1. If d < n, we have succeeded. If d = n, the method failed. We could try again, with a different f or a different x 0.

We expect this method to use the function f at most [image] [image] times, since [image]. This is still not theoretically efficient, but it's better than the simple-minded method of trying all d, [image], checking if d | n, which could take O(n1/2) divisions.

As an example, let's factor n = 551 by this method, with f(x) = x 2+1 mod 551 and x 0 = 2. Here are successive values of x, y, and d = (x − y, n):

x ← f(x) | y ← f(f(y)) | d = (x − y, 551)
5 | 26 | 1
26 | 449 | 1
126 | 240 | 19

Note that 19 and 551/19 = 29 are both prime, so we've factored 551 = 19.29.

11.4.2 Random squares

Among general-purpose factoring methods, the most powerful have been those in the random squares family. The quadratic sieve variant was used to factor integers with 100–129 decimal digits by 1994. A more complicated variant known as the number field sieve factored well-known challenges from RSA Laboratories of 130 digits in 1996 and 140 and 155 digits in 1999. (The 155-digit result in August 1999 was somewhat of a milestone due to the implications for applications relying on the intractability of factoring 512-bit numbers.)

Given composite n, the idea is to locate [image] (mod n); if [image] (mod n), then (x + y, n) is a nontrivial factor of n since n divides x 2 − y 2 = (x+y)(x−y) but does not divide either of the factors. As a special case, if n = pq for distinct primes p and q, then x 2 ≡ a 2 (mod n) for given a ∈ [image] has exactly four solutions, as can be seen by the Chinese remainder theorem. If n = 15, for example, and we manage to stumble upon x = 2 and y = 7 satisfying x 2 ≡ y 2 (mod 15), then (x+y,n) = (9,15) = 3, a nontrivial factor of n.

One method for finding appropriate x and y is to collect pairs [image] mod n) for random ai, seeking a subcollection S for which [image] is a perfect square. In this case, [image] and the square root y of [image] satisfy x 2 ≡ y 2 (mod n). If, in addition, [image] (mod n), then the method has been successful; otherwise a different S can be selected (perhaps requiring the generation of more pairs (ai, bi)).
< previous page | page_269 | next page >


page_27

< previous page | page_27 | next page >

Page 27

Chapter 2
Linear Codes

2.1 Linear codes

In this section we introduce a broad class of codes. In fact, virtually every code we consider will belong to this class. We will be able to bring into play some powerful mathematical tools which will enable us to resolve some of the previously discussed problems of coding theory when applied to codes in this class.

A code C is called a linear code if v + w is a word in C whenever v and w are in C. That is, a linear code is a code which is closed under addition of words. For example C = {000, 111} is a linear code, since all four of the sums

000+000 = 000 111+000 = 111

000+111 = 111 111+111 = 000

are in C. But C 1 = {000, 001, 101} is not a linear code, since 001 and 101 are in C 1 but 001 + 101 is not in C 1.

A linear code C must contain the zero word. For if C is to be linear, then the sum v + v = 0 must be in C by closure under addition. However, as the code C 1 above demonstrates, the zero word being in a code does not guarantee that the code is linear.

Exercises

2.1.1 Determine which of the following codes are linear.

(a) C = {101, 111, 011}

(b) C = {000, 001, 010, 011}

(c) C = {0000, 0001, 1110}

(d) C = {0000, 1001, 0110, 1111}

(e) C = {00000, 11111}

(f) C = {00000, 11100, 00111, 11011}

(g) C = {00000, 11110, 01111, 10001}

(h) C = {000000, 101010, 010101, 111111}

One advantage a linear code has over a nonlinear code is that its distance is easier to find. The distance of a linear code is equal to the minimum weight of any nonzero codeword. Exercise 2.1.4 below requests the easy proof.
< previous page | page_27 | next page >


page_270

< previous page | page_270 | next page >

Page 270

To be more precise, a factor base B = {p 1,...,pt} consisting of the first t primes is selected. Pairs (a,b = a 2 mod n) are retained if b factors over B; such a value b is said to be pt-smooth. Suppose t+1 such pairs (ai, bi) are obtained, where the factorizations are given by [image]. A set S is to be selected so that [image] has only even powers of primes appearing. Note that the t+1 vectors ei = (ei1,...,eit) mod 2 are necessarily linearly dependent over [image], and there exists S so that [image] is the zero vector. The corresponding [image] is then of the desired form.

Example 11.4.1 The method is applied to factor n = 10057. Suppose t = 5 so that the factor base B = {2,3,5,7,11} is selected. The following table lists the ai selected, retaining only those for which bi = a2i mod n factors over the factor base.

The choice a = 7231 is discarded since a 2 mod n does not factor over the factor base. In such cases, the last column may be only a partial factorization.

There are t+1 = 6 valid pairs (ai, bi). Necessarily, there must be some S so that [image] is a perfect square. By inspection, S = {4,5,6} gives x 2 ≡ y 2 (mod n) for x = 3010·4014··4023 mod n = 2748 and y = 27·3·5·7·11 mod n = 7042. Since [image] (mod n), we obtain a nontrivial factor (x+y,n) = 89, and 10057 = 89·113.

A perfect square is also obtained for S = {1,5}. The corresponding values are x = 105·4014 mod n = 9133 and y = 22·3·7·11 = 924. Unfortunately, x ≡ −y (mod n), and no useful information is obtained.

Choosing a larger factor base increases the chances that a given bi will be pt-smooth at the cost of requiring more relations. For a given t, one strategy to improve the chances of obtaining pt-smooth numbers is to select a so that b ≡ a 2 (mod n) is relatively small. The quadratic sieve is a famous example of such a method.

Let n be given and define m = [image]. Define q: → [image] by q(z) = (z+m)2 − n. Note that q(z) ≈ z 2 + 2zm and |q(z)| is small relative to n if |z| is small. The quadratic sieve algorithm takes a = z+m and b = q(z) = a 2 − n for z = 0, ± 1,.... Since b can be negative, -1 is added to the factor base. In addition, note that if a prime p divides b, then a 2 ≡ n (mod p) and n is a quadratic residue modulo p
< previous page | page_270 | next page >


page_271

< previous page | page_271 | next page >

Page 271

(unless p|n). Hence, the factor base need only contain those primes p for which

Example 11.4.2 The value n = 10057 from the last example is factored. Define [image] and q(z) = (z +100)2 – 10057. For a factor base, suppose B = {2,3,11,19} ∪ {–1} is selected (which includes those primes p ≤ 19 such that [image]. The first few z for which q(z) factors over the factor base are listed in the following table.

z | a = z+m | b = q(z) | factorization
0 | 100 | –57 | –3·19
–1 | 99 | –256 | –28
1 | 101 | 144 | 24·32
–3 | 97 | –648 | –23·34
5 | 105 | 968 | 23·112

The relations for z ∈ {–1,–3,5} give [image] (mod n) for x = 99·97·105 and y = 27·32·11. Unfortunately, [image] (mod n) and no useful information is obtained.

The line corresponding to z = 1 gives [image] Note that x = 101 and y = 22·3 satisfy [image] (mod n), and hence a nontrivial factor (x+y,n) = 113 of 10057 has been obtained.

Collecting appropriate (a,b) pairs can be a distributed effort, and trial division is replaced by a more efficient sieving process to test smoothness. The 1994 factoring of a well-known 129-digit challenge involved 600 people and 1600 machines collecting more than 8 million relations over seven months, with a factor base of size 524339 [1]. However, as of 1999, factoring appropriately-chosen n with at least 1024 bits (308 decimal digits) appears to be well out of reach of even the more advanced number field sieve.

11.4.3 Square roots

The previous section suggests a relationship between the problems of factoring and extracting square roots. In fact, the problems are equivalent in a certain precise sense to be discussed in this section.

Recall that a random square factoring method attempts to construct x and y with [image] (mod n); if [image] then a nontrivial factor (x+y,n) of n has been obtained. Clearly, if it was possible to extract all of the square roots for some quadratic residue x 2 in [image], then a nontrivial factor of n can be found. No efficient method in general for uncovering roots is known; however, we wish to consider the implications of such an algorithm.

To be specific, we will consider the case that n = pq for distinct odd primes p and q, although the discussion extends to the general case. By Corollary 11.2.2,

< previous page | page_271 | next page >


page_272

< previous page | page_272 | next page >

Page 272

given a quadratic residue modulo a prime, there are exactly two square roots; the Chinese remainder theorem can be used to show that [image] (mod pq) for [image] has exactly 4 solutions.

We suppose there exists an oracle for SQ ROOT; that is, we assume there is an algorithm returning a square root when presented with a quadratic residue. A factoring attack can then be constructed by selecting random [image] and passing x 2 mod n to the oracle. For each such x, the root y returned by the oracle satisfies [image] with probabilty 1/2; a nontrivial factor (x+y,n) of n is expected in two trials.

The problem of factoring n is said to reduce to the problem of finding square roots. More precisely, given computational problems A and B, we write A ≤ B if A can be solved in polynomial time (in terms of the size of the input) given a polynomial-time algorithm for B. Informally, A ≤ B says that A is no harder than B. The problems are computationally equivalent if A ≤ B and B ≤ A. The same notation will be used in the case that the algorithm involves randomization and has expected running time which is polynomial in the size of the input. In particular, FACTOR ≤ SQ ROOT.

Now consider the problem of finding square roots of a quadratic residue a ∈ Qn given the factors of n. If roots modulo a prime can be extracted, then the 4 roots of a modulo n = pq can be obtained by finding x and y satisfying the congruences

where ar denotes a square root of a modulo r; the roots are then ±x, ±y. Hence, we obtain an efficient algorithm for SQ ROOT if there is an efficient algorithm for extracting roots modulo a prime.

Consider first the case of a prime [image] (mod 4). If [image], then [image] 1 (mod p) by Euler's criterion, and [image] (mod p). Hence, the two square roots of a modulo p are [image]. If both p and q are of this form (in which case n = pq is known as a Blum integer), then we have an efficient deterministic algorithm for finding roots of a modulo n.

It turns out that there is a relatively simple algorithm for extracting roots modulo a prime provided that a quadratic nonresidue is known (in particular, if [image] (mod 8) since [image] in this case; see [63]). However, there is no known efficient deterministic algorithm for finding quadratic nonresidues modulo a prime. Since half of the elements of [image] are quadratic nonresidues, an efficient probabilistic procedure for finding roots modulo a prime can be developed by selecting elements x at random until one is found with [image].

In short, there are efficient methods for extracting roots modulo a prime. It follows that SQ ROOT ≤ FACTOR and hence the two problems are computationally equivalent. The result is discussed again in Section 12.3 in connection with provable security.
< previous page | page_272 | next page >


page_273

< previous page | page_273 | next page >

Page 273

Exercises

11.4.3 This exercise illustrates Pollard's rho factoring algorithm on the toy example n = 391 = 17·23.

(a) Perform Pollard's rho with x 0 = 2 and x i+1 = f(xi) = x2i+1 mod n. (Recall that the process terminates if 1 < (x2i – xi,n) < n. A nontrivial factor is obtained for i = 4.)

(b) Complete the following table:

.

Sketch a rho-diagram for each row of the table, identifying the tail and the cycle.

(c) How should a composite n be chosen to defeat a factoring attack based on Pollard's rho?

11.4.4 Factor n = 5141 using Pollard's rho algorithm, with x 0 = 1 and xi+1 = [image] mod n for i ≥ 0.

11.4.5 Apply the quadratic sieve (as in Example 11.4.2) to factor n = 1081 using an appropriate factor base consisting of integers ≤ 11.

11.4.6 Apply the quadratic sieve (as in Example 11.4.2) to factor n = 24961 using an appropriate factor base consisting of integers ≤ 23.

11.4.7 If n = ab with |a – b| relatively small, then Fermat factorization efficiently recovers nontrivial factors.

• Assume that n is odd. There is a 1 – 1 correspondence between the factorizations of n in the form n = ab, where 0 < a ≤ b, and the representations n = t 2 – s 2, where t and s are nonnegative. The correspondence is given by ab = t 2 – s 2 = (t – s)(t+s), t = (a+b)/2, and s = (b – a)/2.

• If a and b are close together, then s = (b – a)/2 is small and [image]. To factor n, test consecutive values for t starting with [image] until t 2 – n is a perfect square.

Illustrate the method on n = 2881. Fermat factorization is discussed in [50, 74].

11.4.8 Illustrate the procedure for finding square roots of 179 in [image] using the factorization 187 = 11·17.

11.4.9 Devise an efficient test which determines if n is a perfect power (i.e., n = x k for some integers x and k > 1) and, if n is such an integer, finds a partial factorization.

11.4.10 Let n = pq for primes p ≠ q, and let ø = ø (n) = (p – 1)(q – 1). Let COMPUTE-ø denote the problem of finding ø given n. Describe an efficient algorithm for FACTOR which uses an oracle for COMPUTE-ø.
< previous page | page_273 | next page >


page_274

< previous page | page_274 | next page >

Page 274

11.5 Discrete logarithms

Let p be an odd prime and α a generator of [image]. Given [image], the discrete logarithm of β to the base α, written logαβ, is the unique integer x, 0 ≤ x ≤ p – 2, such that [image] (mod p). For example, α = 3 is a generator of [image] and log34 = 2 since 4 ≡ 32(mod 5).

Finding x given (p,αβ) is known as the discrete logarithm problem (DLP). As with integer factorization, it is of course a trivial problem in theory since we may calculate αx for x ≥ 0 until β is found. This is not efficient, requiring 0(p) modular multiplications. No efficient algorithm is known, and several cryptographic schemes rely for security on the supposed intractability of DLP. We shall consider two algorithms which, while not polytime, are more efficient than exhaustive search.

11.5.1 Baby-step giant-step

The baby-step giant-step algorithm is roughly similar to the meet-in-the-middle attack on double-DES, reducing the number of modular multiplications in exhaustive search at the cost of significant storage requirements.

Let m = [image] (mod p), then we may write x = im+j for [image]. Form a table with entries (jαj mod p), 0 ≤ j < m. For each i, 0 ≤ i < m, calculate βα–im mod p and check for a match in the table. On a match, we have [image] (mod p) and logαβ = im+j.

Example 11.5.1 The method is applied to find log62 in [image] (i.e., p = 41, α = 6, and β = 2). Let m = [image] and form a table with entries (j,αj) for 0 ≤ j < 7:

We find α−1 ≡ 7 (mod p) and α−m ≡ 77 ≡ 17 (mod 41). It is now a matter of calculating β(α–m)i mod p until a table entry is found:

A match has been found for i = 3 and j = 5, giving [image] (mod p). Hence, log62 = 26 in [image].

The table requires space for m group elements, built with m – 1 modular multiplications. The giant steps require an inversion and 0(m) modular multiplications, so the running time of the algorithm is [image] modular multiplications. This is better than exhaustive search, but much worse than polynomial time.
< previous page | page_274 | next page >


page_275

< previous page | page_275 | next page >

Page 275

11.5.2 Index calculus

The most powerful methods for finding discrete logarithms are variants of the index calculus algorithm, portions of which resemble random squares algorithms for factoring. A computationally-expensive first stage determines logarithms for elements of a selected factor base B (which need not depend on the specific β for which logαβ is desired). The second stage attempts to find an integer k so that αkβ factors over B; if successful, then logαβ is easily obtained.

For [image], the factor base B = {p1,...,pt} consisting of the first t prime numbers may be selected. In the first stage, random k are selected in an attempt to find values αk mod p which factor over the factor base. For such a k, αk mod [image] for some ei ≥ 0 and hence

Somewhat more than t such congruences are obtained, in the hope that the resulting linear system in the unknowns logα pi has a unique solution.

In the second stage, we seek a value of k for which αkβ mod p factors over B. If successful, then αk β mod [image] for some ei ≥ 0, which implies

Example 11.5.2 The method is illustrated for p = 19 and α = 2, finding logα 17. Suppose the factor base B = {2,3,5} is selected. In this particular example, α ∈ B and logα 2 = 1 is obtained directly. To determine the logs of the remaining two elements of B, we calculate αk mod p for random k, seeking at least two values which factor over the factor base:

The second line is dropped since 14 does not factor over B. This gives the following system of congruences

for the unknowns logα 3 and logα 5. However, these congruences do not determine a unique pair, giving candidates such as the incorrect solution (logα 3 = 4, logα 5 = 7). If the relation α14 mod p = 6 is added, then the new system with the additional congruence 14 ≡ logα2+logα3 (mod p – 1) has a unique solution logα 3 = 13 and logα 5 = 16.
< previous page | page_275 | next page >


page_276

< previous page | page_276 | next page >

Page 276

To find logα 17, we seek k such that αk. 17 mod p factors over B. If k = 5, for example, then α5·17 mod p = 22·3 and logα 17 = (2logα2+logα3 – 5) mod (p – 1) = 10.

The precomputation of logarithms of elements in the factor base is expensive, although collecting suitable relations can be a distributed effort. The first-stage results can be used to find the logarithm of any given β provided that a value of k can be found for which αkβ factors over the factor base. Choosing a larger factor base allows more elements of [image] to be represented as products of elements of B at the cost of a larger system of congruences to be solved.

Sophisticated enhancements of the basic method were used in the 1990s to calculate logs modulo primes of 50–100 decimal digits. As an example, LaMacchia and Odlyzko [54] demonstrated in 1990 that calculation of logs modulo a certain 192-bit (58 decimal digit) prime was quite feasible using an index-calculus variant known as the Gaussian integer method. A system of 288017 relations with 96321 unknowns was reduced to one with 7262 equations and 6006 unknowns and then solved. The database could then be used to obtain individual logarithms with relatively little effort. The problem was of practical interest, since the security in a widely-used identification scheme proposed by Sun Microsystems depended, in part, on the difficulty of extracting logarithms modulo the prime [88].

In 1998, Joux and Lercier calculated discrete logarithms in [image] where p was a 90 digit prime, using a Gaussian integer method. A network of 4 machines (each based on a 180 MHz Pentium Pro, a relatively inexpensive processor) collected 6.7 million equations over a month, obtaining 976062 equations for which every variable.appeared at least twice. The linear system was reduced and then solved in 3 weeks. Selected logarithms required an average of 9 hours each on a single machine.

The number field sieve for factoring integers has been adapted for the log problem. Joux and Lercier used a sieve variant in 1999 to compute logarithms in [image] with p having 100 digits. A machine based on a 450MHz Pentium II collected 2.9 million equations over 8 months, passing the results to a 4-processor (DEC Alpha 500MHz) computer which solved the linear system in 3 weeks. Individual logarithms were calculated in a day. They concluded that the number field sieve ''seems to be better than the Gaussian integer method for computing discrete logarithms modulo primes over 100 digits" [46].

Exercises

11.5.3 Given that α = 5 is a generator of [image], illustrate baby-step giant-step on log54 in [image].

11.5.4 Let p = 41 and α = 6 (a generator of [image]). Illustrate the index calculus method on log613 by completing the following outline.
< previous page | page_276 | next page >


page_277

< previous page | page_277 | next page >

Page 277

(a) Choose factor base B = {2,3,5}. Suppose that αk mod p has been calculated for k ∈ {8,20,16} giving

Verify that the system of congruences does not determine a unique solution (logα2, logα3, logα5).

(b) Add the congruence obtained from α1 mod p = 2·3, and then solve. (The value of logα2 should match that found in Example 11.5.1.)

(c) Find logα 13 by applying (b) to αk·13 mod p for k = 11.

11.5.5 Let p be prime and α a generator of [image]. Show that the least significant bit of x can be efficiently determined from αx mod p.

11.5.6 Bach [2] discusses relationships between factoring and finding logarithms. In particular, this exercise shows that an oracle giving x for a x ≡ b (mod n) can be used in a probabilistic factoring attack on composite n.

Consider the case n = pq for odd primes p ≠ q. Let λ = 1cm(φ(p),φ(q)).

(a) Show that [image] (mod n)} is a proper subgroup of [image]. Hence, at least half the elements of [image] are outside K.

(b) Let [image], and suppose that ax ≡ 1 (mod n) for some 0 ≠ x (known as an exponent for a). Show that there exists 0 < k < log2 x such that [image] is a nontrivial square root of 1(that is, [image] and [image].

(c) Conclude that ( [image] + 1, n) is a nontrivial factor of n.

A factoring attack uses the oracle for a x ≡ b (mod n) to find an exponent for a. Although φ(n) is unknown, there must be some r among the first log2 n primes such that (r,φ(n)) = 1. For such an r, the oracle returns a solution y of (a r)y ≡ a (mod n), and x = ry – 1 is an exponent for a.

11.6 Notes

Much of the material in this chapter is classical, for which many good introductory books exist, e.g., Rosen [74] and Koblitz [50]. Chapters 2 and 3 of Menezes, van Oorshot, and Vanstone [63] concisely covers some of the relevant material, and includes many references. Koblitz includes several examples of calculating time estimates (in terms of bit operations) for arithmetic, a topic which is less-commonly covered in number theory books.
< previous page | page_277 | next page >


page_278

< previous page | page_278 | next page >
Page 278 This page intentionally left blank.
< previous page | page_278 | next page >


page_279

< previous page | page_279 | next page >

Page 279

Chapter 12
Public-key Cryptography

The fundamental feature which distinguishes public-key from symmetric-key schemes is the separation of encryption and decryption capabilities. To be slightly more precise, the key k in a public-key scheme is written as a key-pair k = (e,d), where e is used for encryption and d is used for decryption. In this context, e is the public key and need not be secret; d is the private key, and is held only by those needing to decrypt messages. To be useful as an encryption scheme, it must be computationally infeasible for an adversary in possession of e and ciphertext c to find m so that Ee(m) = c.

The idea of public-key schemes was introduced by Diffie and Hellman in 1976 to exchange keys over an unsecured but authenticated channel. In the context of Figure 10.1, the understanding is that Alice and Bob are certain of the origin and integrity of communication over the unsecured channel, but there may be eavesdropping (passive attacks) by an adversary Eve. The problem is to maintain secrecy (even though Eve sees the entire conversation) without relying on the secure channel portion of the figure to transfer keys or other information.

The following key agreement is proposed: Alice and Bob choose a prime p and a generator α of [image] p and α may be public. In private, Alice chooses a random a, 1 ≤ a < p, and transmits αa to Bob (in full view of Eve).1 Similarly, Bob chooses a random b, 1 ≤ b < p, and sends αb to Alice. Alice and Bob may efficiently calculate (αb)a mod p and (αa)b mod p, respectively, arriving at the shared secret k = αab mod p from which a common key may be derived for subsequent use in a symmetric-key scheme such as DES to achieve confidentiality.

Eve, of course, would also like to have the shared secret k. Her quest is known as the Diffie-Hellman problem (DHP), related to the discrete log problem (DLP):

DHP: Given prime p, generator α of [image], αa, and αb, find αab.

DLP: Given prime p, generator α of [image], and αx, find x.

1For brevity, the operator ''mod p" will often be omitted if the meaning is clear from context. Here, for example, we mean that αa mod p is transmitted.
< previous page | page_279 | next page >


page_28

< previous page | page_28 | next page >

Page 28

Exercises

2.1.2 Show that C = {0000, 1100, 0011, 1111} is a linear code and that its distance is d = 2.

2.1.3 Find the distance of each linear code in Exercise 2.1.1. Check answers with Exercise 1.11.12.

2.1.4 Prove that the distance of a linear code is the weight of the nonzero codeword of least weight.

As we will see in the following sections, linear codes are rather highly structured and have many other advantages over the arbitrary codes discussed so far. Here are some problems, tedious to settle in general, but relatively easy for linear codes:

1. For a linear code, there is a procedure for MLD that is simpler and faster to use than the one described earlier (certain linear codes with even more structure have very simple decoding algorithms).

2. Encoding a linear code is faster and requires less storage space than for arbitrary non-linear codes.

3. The probabilities θp(C, v) are straightforward to calculate for a linear code.

4. It is easy to describe the set of error patterns that a linear code will detect.

5. It is much easier to describe the set of error patterns a linear code will correct than it is for arbitrary non-linear codes.

The most important tools and techniques for studying linear codes come from linear algebra. In this and the next several sections we will review some basic facts from linear algebra and attempt to show their relevance to coding theory. Most proofs not depending on scalar products in Kn are exact replicas of the proofs in Rn, and hence are omitted.

Recall that we defined a vector space (over K), as consisting of scalars (K) and a set of vectors, or words, Kn, together with the operations of vector addition and scalar multiplication, which satisfy the ten properties listed in Section 1.7. A nonempty subset U of a vector space V is a subspace of V if U is closed under vector addition and scalar multiplication; that is, if v and w are in U, then v + w and av are in U for any scalar a. In particular, since the only scalars in K are 0 and 1, U is a subspace of Kn if and only if U is closed under addition. Therefore C is a linear code if and only if C is a subspace of Kn. Over the next few sections we shall use the knowledge of subspaces to dramatically improve our techniques for encoding and decoding.

2.2 Two important subspaces

We consider two subspaces of the vector space Kn which will provide two interesting examples of linear codes and will be vital in future developments. Defini-
< previous page | page_28 | next page >


page_280

< previous page | page_280 | next page >

Page 280

It is clear that DHP ≤ DLP; that is, DHP can be solved in polynomial time given a polytime oracle for DLP. In some special cases, it is known that DLP ≤ DHP, but the general case remains an open question [63]. The presumed intractability of DHP is the basis for the security of the Diffie-Hellman key agreement.

Public-key schemes can be used to address some difficulties inherent in symmetric-key schemes. The Diffie-Hellman key agreement suggests that secrecy can be maintained even without a secure channel for exchanging keys. (It is important to note the assumption of an authenticated channel—the agreement is not secure against an active adversary, as illustrated in Section 12.5.) Provided that the public keys can be authenticated, key distribution does not require a trusted courier. Since secrecy of the public key is not required, authentication is presumably easier than exchanging symmetric keys.

Public-key reduces the key management problem. If there are n users of a cryptosystem, then [image] keys are required in a symmetric-key scheme compared with 2n in a public-key scheme, a significant difference if n is large.

Digital signatures (discussed in the following sections) are another area where public-key schemes are attractive. Alice should be able to sign a message in such a way that not only is Bob convinced that the message originated from Alice, but Bob can convince a third party of the same. The basic problem with using symmetric-key schemes for signatures is that Bob and Alice have the same capabilities. A trusted third party can be used to enable signatures, but public-key schemes provide a more mathematically elegant and practical solution.

12.1 One-way and hash functions

Two cryptographic primitives are examined in this section, trapdoor one-way functions (the fundamental notion in public-key schemes), and cryptographic hash functions (commonly used in signature schemes).

One-way functions

Informally, a function f : M → C is one-way if f(m) is easy to compute for all m ∈ M, and for essentially all c ∈ C it is computationally infeasible to find m so that f(m) = c. The function described in Section 10.3.2 for Unix passwords is believed to be one-way (under some ceiling on computational ability), storing f(userid, password) rather than the password itself.2 Another candidate is discrete exponentiation. Let p be prime and α a generator of [image]. The function [image] defined by f(a) = αa mod p is believed to be one-way.

2There was sufficient confidence in the scheme that the stored values were visible to all the users of the system, inviting password-cracking analysis. Needless to say, this turned out to be a bad idea, since users often select predictable passwords. Modern systems tend to demand better passwords and have made the stored values private.
< previous page | page_280 | next page >


page_281

< previous page | page_281 | next page >

Page 281

It is not known if one-way functions exist. Public-key schemes certainly assume that various functions, if not one-way, will be effectively one-way for the life of the specific security objective. A number of candidates for use in public-key schemes are discussed in the following sections.

A one-way function is trapdoor if it has the property that given some extra information, it becomes feasible to find m so that f(m) = c for any given c. The trapdoor one-way function is the basic tool in public-key cryptography.

Example 12.1.1 (applications of trapdoor functions) Confidentiality. Each user A picks their own trapdoor one-way function fA and makes it public. To transmit a confidential message m to A, the sender forwards c = f A(m). Only A has the trapdoor information allowing f to be inverted—even the sender cannot recover m from c. No secret information needs to be shared in advance, although it is necessary for the sender to have a means of authenticating the public key.

Non-repudiation. The analogue of a hand-written signature is desired: a message m from Alice is to be signed in such a way that Bob can convince any third party that m originated with Alice. Assume that m has some prescribed degree of redundancy. If Alice's trapdoor one-way function is f A : M → M, then Alice sends [image]. Bob computes m = f A(s) and (m,s) is the signed message. Although an adversary can calculate m = f A(s) for chosen s, forgeries are thwarted by the assumed redundancy in m. If secrecy is desired, Alice could send c = f B(s) rather than s directly, where f B is Bob's trapdoor one-way function.

Cryptographic hash functions

A mapping H : X → Y which is not one-to-one is known as a hash function. For a given x ∈ X, the value H(x) is called a hash, and serves as an ''identifier" for x. By definition, there are necessarily collisions—values x 1 ≠ x 2 with the same hash. Typical design goals for a hash function include spreading conditions on the collisions and efficient calculation of the hash values.

Public-key cryptosystems tend to be relatively computationally intensive, and signing long messages is a slow procedure. In practice, a hash function is used to create a message digest of the message, and it is the digest that is signed. In order for this to be secure, additional features of the hash function are needed.

Definition 12.1.2 A cryptographic hash function is a mapping H : {0, 1}* → {0, 1}n such that:

1. H can be efficiently computed;

2. (preimage resistance) for essentially all y ∈ {0, 1}n, it is computationally infeasible to find x ∈ {0, 1}* such that H(x) = y;

3. (collision resistance) it is computationally infeasible to find x 1 ≠ x 2 ∈ {0, 1}* such that H(x1) = H(x2).
< previous page | page_281 | next page >


page_282

< previous page | page_282 | next page >

Page 282

It is not known if cryptographic hash functions exist (since 1 and 2 imply that such a function is one-way); however, there are a number of candidates used for data integrity and signature schemes.

Example 12.1.3 (application to signatures) To sign a message m, Alice computes and signs the hash H(m). The message m and the signature on H(m) are sent to Bob, who computes H(m) and verifies the signature. A scheme where the message itself is required in the verification process is known as a digital signature scheme with appendix.

If the hash function H is not preimage resistant, then an adversary (perhaps Bob) given a valid signature for H(m) can forge Alice's signature by finding a message m′ with H(m′) = H(m), obtaining a valid signature for m′.

If m ≠ m′ giving collisions can be found, then Alice can cheat by signing m and later claiming to have signed m′. A variation on this occurs if an adversary finds such a collision and convinces Alice to sign one of the messages.

In the example, the signature presumably protects the integrity of the hash computed by Alice. In general, if the hash value can be protected by some mechanism, then a hash function can be used to verify that the corresponding data has not been modified. In this context, an unkeyed hash function is called a modification detection code (MDC). A keyed hash function providing data origin authentication using a secret key is called a message authentication code (MAC); the CBC-MAC described in Section 10.3.2 is an example.

Several hash functions have been proposed which are built on block ciphers. As noted, CBC-MAC is a keyed hash function, while the following two examples are unkeyed.

Example 12.1.4 (Matyas-Meyer-Oseas hash) Let E be a block cipher, Ek : {0, 1}n → {0, 1}n, with keyspace K. Let g : {0, 1}n → K be a (non-secret) function, and let H0 be a (non-secret) initial value in {0, 1}n. The mapping H : {0, 1}* → {0, 1}n is defined as follows:

1. Break x into n-bit blocks, say x = x 1...xt.

2. Let H i = E g(H i–1)(x i) ⊕ x i, 1 ≤ i ≤ t. Then H(x) = H t.

A schematic appears in Figure 12.1(a).

Roughly speaking, if the roles of the xi and Hi–1 are interchanged, then the following proposed hash function is obtained. Note however, that the splitting of x differs.

Example 12.1.5 (Davies-Meyer hash) Let E be an n-bit block cipher using k-bit keys. Let H 0 be a (non-secret) initial value in {0, 1}n. The mapping H : {0, 1}* → {0, 1}n is defined as follows:

1. Break x into k-bit blocks, say x = x 1...xt.

2. Let H i = E xi(H i–1) ⊕ H i–1, 1 ≤ i ≤ t. Then H(x) = H t.

A schematic appears in Figure 12.1(b).
< previous page | page_282 | next page >


page_283

< previous page | page_283 | next page >

Page 283

Figure 12.1: Unkeyed hash functions based on block cipher E.

The message digest produced by an unkeyed hash function must be of sufficient length to prevent birthday attacks (random searches for collisions). For n-bit hash functions, it is expected that collisions can be found in at most 2n/2 operations. In particular, neither of the hash functions in Figure 12.1 is collision-resistant if E = DES. Two well-known hash functions, the Secure Hash Algorithm (SHA-1) and MD5 (both based on MD4, number 4 in a series of Message Digest algorithms from Rivest) have digest bitlength 160 and 128, respectively.

Exercises

12.1.6 Suppose g : {0, 1}* → {0, 1}n is a collision-resistant hash function. Let h be defined by

where '||' denotes concatenation. Give an informal argument to show that h is an (n + 1)-bit hash function which is collision resistant but not preimage resistant. (The example is from [63, Note 9.20].)

12.1.7 This exercise is adapted from [63, Example 9.64], illustrating that some care must be taken when constructing a MAC from an MDC.

Suppose that h is an MDC defined in an iterated fashion on message x = x 1...x t by H i = f(H i–1,x i), h(x) = H t, where H 0 is a given initial value. It is proposed that h can be converted into a MAC by prepending a secret key k so that the MAC on a message x is M = h(kx). Show that knowledge of an (M, x) pair can be used to forge a MAC on xy, even though k is unknown.

12.1.8 Suppose p and q are primes such that p′ = 2p+1 and q′ = 2q+1 are also prime, and let n = p′q′. Suppose α is an element of order 2pq in [image] and let h : be defined by h(x) = αx mod n. If h(x 1) = h(x 2) = h(x 3), show that n can be efficiently factored under suitable conditions on the collisions xi.

Illustrate the process with the toy modulus n = 77 and α = 2, where h(9) = h(69) = h(129). (This exercise is adapted from [86, Exercise 7.4].)
< previous page | page_283 | next page >


page_284

< previous page | page_284 | next page >

Page 284

12.2 RSA

Proposed in 1977 by Rivest, Shamir, and Adleman, the RSA cryptosystem is widely-used for encryption and signature schemes, with security based on the presumed intractability of the integer factorization problem. The RSA function itself is defined as follows.

1. Let p ≠ q be large primes, and define n = pq and ø = (p − 1)(q − 1).

2. Choose random encryption exponent e, 1 < e < ø, so that (e ø) = 1.

3. Use the Euclidean algorithm to find decryption exponent d, 1 < d < ø, so that ed ≡ 1 (mod ø).

4. Define f: by f(m) = m e mod n.

The RSA function f is believed to be one-way with trapdoor d. Given d, recovery of m from c = f(m) can be done efficiently using the property ed ≡ 1 (mod ø). To see this, first suppose that p ′ m. Note that ed = 1 + kø = 1 + k(p − 1)(q − 1) for some k, and by Fermat's theorem m p – 1 ≡ 1 (mod p) so that m 1 + k(p – 1)(q – 1) ≡ m (mod p). If p | m, then the last congruence holds trivially.

This shows that

m 1 + k(p – 1)(q – 1) ≡ med ≡ m (mod p).

By a similar argument med ≡ m (mod q), and hence med ≡ m (mod n) since p and q are distinct primes. It follows that

c d ≡ (m e)d ≡ m (mod n),

efficiently recovering m from c by modular exponentiation.

Inverting f without possessing d is the RSA problem: given a product n = pq of distinct primes, a positive integer e relatively prime to (p − 1)(q − 1), and an integer c, find m such that m e ≡ c (mod n). If n can be factored, then d may be computed and m obtained efficiently. However, factoring n for appropriately chosen p and q is believed to be a hard problem.

Exercise 12.2.12 shows that an efficient method to factor n is obtained if d can be computed. Hence, the problem of factoring n and the problem of computing d from n and e are computationally equivalent. The equivalence provides some evidence that the RSA problem may be as hard as factoring, but this remains an open question.3

In the RSA cryptosystem, (n,e) is the public key and d is the private key. Encryption of m ∈ [0, n − 1] gives c = me mod n; decryption is m = cd mod n.

Example 12.2.1 (RSA with toy parameters) Suppose p = 7 and q = 13 are used in RSA key generation. Then ø = (p − 1)(q − 1) = 72, and we may choose encryption exponent e = 5. The condition ed ≡ 1 (mod ø) gives decryption exponent

3Boneh and Venkatesan [15] provide evidence that breaking low-exponent RSA (that is, RSA with small encryption exponent e) cannot be equivalent to factoring.
< previous page | page_284 | next page >


page_285

< previous page | page_285 | next page >

Page 285

d = 29. The public key is (n = 91, e = 5) and the private key is d = 29. Messages m consist of integers in [0,91). If m = 23, then c = me = 235 mod 91 = 4 and cd mod n = 429 mod 91 = 23 can be verified using repeated squaring.

An RSA encryption scheme passes some messages unconcealed; that is, there are messages m with c = me mod n = m. The messages m ∈ {0, 1, n − 1} are always unconcealed. Note that me ≡ m (mod n) if and only me ≡ m (mod p) and me ≡ m (mod q). Solutions of me ≡ m (mod p) satisfy m ≡ 0 (mod p) or m e – 1 ≡ m (mod p); there are 1 + (e − 1, p − 1) such solutions (Exercise 11.1.22). The Chinese remainder theorem can be applied to show that the number of unconcealed messages is (1 + (e − 1, p − 1))(1 + (e − 1, q − 1)). In particular, there are always at least 9 unconcealed messages since e, p, and q are odd. Example 12.2.1 has 15 unconcealed messages; m = 8 is one such message. Exercise 12.2.3 illustrates the extreme case where every message is unconcealed.

Although known- and chosen-plaintext attacks generally do not apply to public-key encryption, there are scenarios where a chosen-ciphertext attack is a concern. Let c 1 and c 2 be RSA encryptions of m 1 and m 2, respectively. Then [image] (mod n), giving c = c 1 c 2 mod n as the ciphertext corresponding to m = m 1 m 2 mod n. An adversary Eve may be able to exploit this multiplicative property by concealing the target ciphertext c in [image] and convincing the private-key holder Alice to provide decryption of [image]. Eve chooses [image] and passes [image] to Alice for decryption, obtaining [image] mod n. Since [image] (mod n), Eve recovers [image] mod n. Structural requirements on plaintext messages may defeat the attack.

The modulus selected during key generation must be sufficiently large to circumvent factoring attacks. In 1996, Menezes et al. [63] recommended a 768-bit or larger modulus, with 1024-bit moduli for long-term security. Lenstra and Verheul presented an argument for longer moduli in 1999, concluding that ''768-bit RSA keys will soon no longer offer security comparable to the security of the DES in 1982" [55]. Additional restrictions on the factors may be imposed due to various factoring attacks; for example, p and q should be roughly the same bitlength but not so that | p – q | is particularly small.

RSA signature scheme (with message recovery)

RSA may be used in digital signatures, interchanging the functions of encryption and decryption as outlined in Example 12.1.1. Alice's signature on m is s = md mod n, where d is Alice's private key. On receiving s, Bob computes se mod n = m using Alice's public key (n,e), obtaining signed message (m,s).

In order to provide a mechanism to detect invalid messages, structural requirements are placed on messages m. A forger can choose s (and obtain corresponding m = se mod n using Alice's public key), but has little control over the structure of

< previous page | page_285 | next page >


page_286

< previous page | page_286 | next page >

Page 286

the message m. Bob accepts signed message (m,s) only if the prescribed structure of m is present.

To reduce signature-verification or encryption times, it is desirable to choose a small exponent e, although exercises 12.2.4 and 12.2.5 illustrate some concerns with such a choice. Similarly, a private exponent d which is relatively small would improve decryption and signature-generation times, but the concern is that the key itself may be leaked. Under fairly general conditions, Wiener [95] showed that the private key can be recovered if d is small relative to n, using a continued-fractions method (see Exercise 12.2.7).

In practice, a scheme based on RSA would be implemented in a more sophisticated fashion than the basic versions described here. Padding of the message before encryption is commonly used to defeat chosen-ciphertext and other attacks; at a minimum, padding with some randomness has the advantage that repeated encryptions of a fixed message will likely give different ciphertexts [4]. Boneh [12] surveys two decades of attacks, noting that ''none of them is devastating. They mostly illustrate the dangers of improper use of RSA. Indeed, securely implementing RSA is a nontrivial task."

Exercises

12.2.2 Alice chooses p = 31, q = 47, and e = 77 for use in RSA.

(a) What is Alice's private key?

(b) Encrypt the message m = 3 using Alice's public key.

(c) Verify your answer to (b) by decrypting the ciphertext to recover m.

12.2.3 Show that every message is unconcealed if p = 5, q = 17, and e = 33 are chosen in RSA.

12.2.4 A small encryption exponent in RSA is desirable for encryption efficiency. Suppose three users have e = 3 and RSA moduli n 1,n 2, and n 3, resp. An adversary intercepts ciphertext ci = me mod ni, 1 ≤ i ≤ 3, from the common message m. If the moduli are pairwise relatively prime, show how Gauss' algorithm can be used to recover the message m.

This is an example of a protocol failure in RSA. A larger exponent (such as e = 216 + 1 with only a few 1s in its binary representation to maintain efficiency) is one countermeasure; generating a random bitstring and appending to the message for each encryption, known as salting, is another.

12.2.5 (Related messages and RSA) Coppersmith et al. [24] describe an effective attack against low-exponent RSA in the case that the underlying plaintext messages satisfy a known linear relationship. Consider the case that the encryption exponent is e = 3 and that ciphertexts c 1 and c 2 correspond to messages m and m + 1 for some (unknown) m; i.e., c 1 = m 3 mod n and c 2 = (m + 1)3 mod n.
< previous page | page_286 | next page >


page_287

< previous page | page_287 | next page >

Page 287

(a) Verify that m can be recovered from the ciphertexts by

(b) Note that x − m is a common factor of x 3 − c 1 and (x + 1)3 − c 2. Establish the formula in (a) by adapting the Euclidean algorithm to find ((x + 1)3 − c2, x3 − c1). Verify that the algorithm will indeed return a linear polynomial.

12.2.6 (Partial key-exposure) Let n = pq where p and q are primes with 5 ≤ p < q < 2p. Let integers e and d satisfy 1 < e,d < ø(n) and ed ≡ 1 (mod ø(n)).

(a) Since ed ≡ 1 (mod ø(n)), there exists an integer k with ed − kø(n) = 1. Prove that 1 ≤ k ≤ e.

(b) Let [image]. Prove that [image].

(c) If e = 3, then prove that k = 2.

(d) Devise an efficient algorithm for computing the leftmost-half bits of d, given only n and e = 3. (More precisely, the algorithm should narrow the possible values to one or two.)

The exercise is adapted from [12].

12.2.7 (Small decryption exponent) Let n = pq for odd primes p and q satisfying p < q < 2p. Let integers e and d satisfy 1 < e,d < ø(n) and ed ≡ 1 (mod ø(n)). Suppose that [image], and let k be such that ed − kø(n) = 1.

(a) Prove that [image] that is, n is a good approximation of ø(n).

(b) Prove that [image] is a good approximation of [image]. More precisely, prove that [image] [image].

(c) Using the fact below, devise an efficient algorithm for computing d, given only n and e.

Fact. Let 1 ≤ x 0 < y 0 for integers x 0 and y 0. The number of pairs (x,y) with 1 ≤ y < y 0 such that [image] is bounded by 2log2 y 0. Moreover, all such pairs can be found efficiently (as convergents of the continued fraction expansion of [image]; see Hardy and Wright [42]).

The attack is discussed in [12, 14] and is due to Wiener [95].

12.2.8 (Fault analysis attack [13, 47]) Suppose that a smartcard is using the Chinese remainder theorem for RSA decryption; that is, decryption of ciphertext c is performed by computing mp = cd mod(p−1) mod p and mq = cd mod(q−1) mod q, and then finding m, 0 ≤ m < n, such that m ≡ m p (mod p) and m q (mod q).

(a) Prove that the process provides the correct decryption (i.e., m = cd mod n).

(b) Suppose that the smartcard can be induced to compute mp incorrectly (and mq correctly). Let m′ be the resulting (incorrect) decryption of c. Show how this information can be used to factor n.
< previous page | page_287 | next page >


page_288

< previous page | page_288 | next page >

Page 288

12.2.9 A cycling attack against RSA finds the first positive integer k such that [image] (mod n).

(a) Show that such an integer k exists. Then show that an adversary recovers the message with [image] (mod n).

(b) The attack can be generalized. Let u be the smallest positive integer such that [image]. Show that either a nontrivial factor of n has been obtained or the method has reduced to the basic cycling attack (i.e., u = k).

Cycling attacks are not likely to succeed for n = pq if p − 1 and q − 1 have large prime factors [63]. The generalized attack is expected to terminate before the basic cycling, and can be viewed as a factoring attack (and so not likely to be successful if factoring is intractable).

12.2.10 Verify that RSA key generation could use the universal exponentλ(pq) = lcm(p − 1, q − 1) in place of ø(n). Using λ may result in a smaller decryption exponent d.

12.2.11 RSA key generation requires large primes p and q. Suppose, in fact, that a probabilistic primality test produced a ''probably prime" p that was actually composite. Consider the case that p = p 1 p 2 where p 1 ≠ p 2 are primes distinct from q. Key generation will obtain e and d from ø(p,q) = (p − 1)(q − 1) rather than ø(n) = (p 1 − 1)(q − 1)(q − 1).

(a) Note that λ = 1cm(p 1 − 1, p 2 − 1, q − 1) | ø(n). Show that if λ | ø(p,q) then encryption and decryption will succeed.

(b) Illustrate (a) with p = 15 and q = 7, finding λ, ø(n), and ø(p,q).

(c) Suppose p = 21 and q = 5. Show that λ ′ ø(p,q). Find d for encryption exponent e = 3, and exhibit a message m with cd mod n ≠ m where c = me mod n.

12.2.12 This exercise shows that possession of the decryption exponent d in RSA allows n to be factored efficiently. The main idea is that nontrivial square roots modulo n of 1 can be found.

Since ed ≡ 1 (mod ø), it follows that m ed – 1 ≡ 1 (mod n) for all [image]. Write ed − 1 = 2s t where t is odd. The proof consists of showing that, for at least half of all [image] there exists r ∈ [1,s] such that [image] (mod n) and [image] (mod n); for such an m [image] is a nontrivial factor of n.

An [image] fails to lead to a factor if

m t ≡ 1 (mod n) or [image] (mod n) for some 0 ≤ r < s.

Determining the number solutions to these congruences is the main work. Let p − 1 = 2i p′ and q − 1 = 2j q′ for p′ and q′ odd. Recall that ax ≡ b (mod n) has a solution x iff (a,n) | b in which case there are exactly (a,n) solutions (Exercise 11.1.20).
< previous page | page_288 | next page >


page_289

< previous page | page_289 | next page >

Page 289

Case mt ≡ 1 (mod n). First consider the congruence modulo p. Let α be a generator of [image]. We seek the number of solutions x of (αx)t ≡ 1 (mod p).

(a) Show that p′ divides t.

(b) Note that [image] (mod p). Show that there are exactly p′ solutions.

(c) Similarly, the number of solutions of mt ≡ 1 (mod q) is q′. Use the Chinese remainder theorem to show that the number of solutions of mt ≡ 1 (mod n) is p′q′.

Case [image] (mod n ). As above, the congruence is first is examined modulo p and then modulo q. We seek the number of solutions [image] (mod p).

(d) Note that αp−1/2 ≡ − 1 (mod p) so that [image] (mod p) iff [image] [image] Show that there are solutions iff r < i, in which case there are exactly 2′ p′ solutions

We may assume without loss of generality that i ≤ j. The Chinese remainder theorem may be applied, showing that the number of solutions to [image] (mod n) is 22r p′q′ if r < i and 0 otherwise.

An upper bound on the number of m [image] satisfying [image] (mod n) for some 0 ≤ r < s is then [image]. Necessarily, an upper bound on the number of m [image] which fail to lead to a factorization of n is then

Since 22i p′q′ ≤ 2i p′ ≤ 2i p′ 2j q′ = (p – 1)(q – 1) = φ(n), it follows that

Necessarily, for at least half the elements m [image] there exists 1 ≤r≤ s such that [image]. To factor n, select random m [image] and determine if r exists; if so, calculate [image]. A nontrivial factor of n is expected in two trials.

12.2.13 If it is known that ciphertext c = me mod pq was obtained for m ∈[0,p), then decryption can be done in [image], recovering the message as m = cd mod n = cd mod p−1 mod p. Shamir [78] proposed an unbalanced RSA which selects primes p < q of very different sizes, say 500 and 4500 bits, respectively, with plaintext messages restricted to the interval [0, p). Factoring attacks on n = pq are defeated, but decryption is as fast as RSA with a 500-bit modulus.

Show that unbalanced RSA is subject to a complete break by a chosen-ciphertext attack.
< previous page | page_289 | next page >


page_29

< previous page | page_29 | next page >

Page 29

tions and results will be stated for an arbitrary vector space, then interpreted for Kn.

The vector w is said to be a linear combination of vectors v 1, v 2,..., vk if there are scalars a 1, a 2,..., ak such that

w = a 1 v 1 + a 2 v 2 +...+ ak vk.

The set of all linear combinations of the vectors in a given set S = {v 1, v 2,...,vk} is called the linear span of S, and is denoted by 〈S〉. If S is empty, we define 〈S〉 = {0}.

In linear algebra it is shown that for any subset S of a vector space V, the linear span 〈S〉 is a subspace of V, called the subspace spanned or generated by S. For the vector space Kn, we have a very simple description of 〈S〉 which is stated in the next theorem. Since 〈S〉 is a subspace, in Kn we call 〈S〉 the linear code generated by S.

Theorem 2.2.1 For any subset S of Kn, the code C = 〈S〉 generated by S consists precisely of the following words: the zero word, all words in S, and all sums of two or more words in S.

Example 2.2.2 Let S = {0100, 0011, 1100}. Then the code C = 〈S〉 generated by S consists of

0000, 0100, 0100+0011 = 0111, 0100+0011+1100 = 1011,

1100, 0011, 0100+1100 = 1000, 0011+1100 = 1111;

that is, C = 〈S〉 = {0000, 0100, 0011, 1100, 0111, 1000, 1111, 1011}.

Exercises

2.2.3 For each of the following sets S, list the elements of the linear code 〈S〉.

(a) S = {010, 011, 111}

(b) S = {1010, 0101, 1111}

(c) S = {0101, 1010, 1100}

(d) S = {1000, 0100, 0010, 0001}

(e) S = {11000, 01111, 11110, 01010}

(f) S = {10101, 01010, 11111, 00011, 10110}

If v = (a 1, a 2,...,an) and w = (b 1, b 2,..., bn) are vectors in Kn, we define the scalar product or dot product v·w of v and w as

v·w = a 1 b 1+a 2 b 2+...+an bn.

Note that v·w is a scalar, not a vector. For instance, in K 5,
< previous page | page_29 | next page >


page_290

< previous page | page_290 | next page >

Page 290

12.2.14 A ''dark side" of black-box cryptography is examined in CRYPTO '96 [102]. The black-box is contaminated with a mechanism that allows the manufacturer to extract secrets, but protects against detection and attacks by others. The mechanism is known as a Secretly Embedded Trapdoor with Universal Protection (SETUP).

The authors' most straightforward SETUP concerns RSA. The idea is to hide enough information in an RSA public exponent e so that the attacker can factor n. The black-box normally selects primes p ≠ q and then produces an RSA key (n = pq, e, d), The process is contaminated using an attacker's key (n′, e′), attempting to select a prime p for which e = pe′ mod n′ satisfies (e,φ(n)) = 1.

(a) The authors of [102] left a few details to the reader. How does the attacker factor n from the public key (n,e), and what assumptions have been made?

(b) Speculate on the likeliness that the SETUP will be discovered, perhaps based on key-generation times or the form of the keys produced.

The SETUP described will not be effective in cases where a small e is required. The authors give a "Pretty Awful Privacy" SETUP which is similar to PGP (Pretty Good Privacy [37, 104]), but hides information about p in the modulus n. See also [103] and Exercise 12.5.6.

12.3 Provable security

The security in RSA is said to be based on the presumed intractability of factoring; however, systematic recovery of plaintext from ciphertext in RSA is not known to be as difficult as factoring. It is possible that the integer factorization problem may be intractable but breaking RSA is easy.

In 1979, Rabin [69] introduced a scheme which is provably secure in the sense that recovery of plaintext from some given ciphertext is as difficult as solving a computational problem believed to be intractable. Provable security clarifies and minimizes assumptions, and allows a more precise definition of security. It should be emphasized that the proof is relative to the assumptions about the related computational problem—there may be considerable evidence that problems such as factoring and discrete logs are hard, but none are proven computationally infeasible.

The Rabin scheme is similar to RSA in that the encryption function in both is of the form f(m) = me mod pq. In RSA, the operation is invertible; however, Rabin uses e = 2 and computation of square roots recovers the possible choices for m. To be precise, the Rabin scheme chooses primes p ≠ q, forming public key n = pq and private key (p,q). The encryption function f : is given by f(m) = m 2 mod n. Given ciphertext c, decryption proceeds by solving c=m 2 mod n for the four possible roots using the techniques described in Section 11.4.
< previous page | page_290 | next page >


page_291

< previous page | page_291 | next page >

Page 291

Example 12.3.1 Consider the Rabin scheme with toy parameters p = 31 and q = 41. The message m = 814 is encrypted with the public key n = pq = 1271, obtaining c = f(m) = m 2 mod n = 8142 mod 1271 = 405. To decrypt, the private key (p = 13, q = 41) is used to extract square roots, solving the congruences

m2 ≡ 405 ≡ 2 (mod 31)

m2 ≡ 405 ≡ 36 (mod 41)

for m. Section 11.4 shows that this can be done efficiently. In this toy problem, p ≡ 3 (mod 4) so that the roots modulo p are given by m ≡ ±2(p+1)/4 ≡ ±8 (mod 31); the second congruence gives m ≡ ±6 (mod 41) by inspection. Gauss' algorithm 11.1.6 may be applied, obtaining (at most) four possibilities for m:

(Note that only the first two need to be solved; the roots are then ±m 1, ±m 2.) Four distinct candidates mi are obtained for m. Unless structural requirements are placed on messages, the receiver is unable to determine that m 3 mod n = 814 is the original message.

To understand the security of the scheme, consider the problem of interest to an adversary:

RABIN: Given n = pq and c = m 2 mod n, find x so that c ≡ x 2 (mod n).

This is the square root problem discussed in Section 11.4. If p ≡ q ≡ 3 (mod 4), then n = pq is a Blum integer and square roots can be extracted in polynomial time. In the example, q ≡ 1 (mod 8), and n is not of the special form. There is no known polynomial-time deterministic algorithm for solving x2 ≡ c (mod q) when q ≡ 1 (mod 8); there is, however, the randomized method discussed in Section 11.4 which runs in expected polynomial time. Hence, RABIN ≤ FACTOR; that is, given an oracle for FACTOR that runs in expected polynomial time, then we may solve RABIN in expected polynomial time.

To see that FACTOR ≤ RABIN, recall that if x2 ≡ y2 (mod n) and x ≢ ± y (mod n), then (x+y,n) is a nontrivial factor of n. Choose [image] at random, and compute c = x 2 mod n. The oracle for RABIN is queried, and a square root y of c is returned in expected polynomial time. With probability 1/2, the root y satisfies y ≢ ± x; in this case, (x+y,n) is one of the prime factors of n.

Hence, breaking the Rabin system is equivalent to factoring n. Note, however, the argument also shows that the scheme is not secure against a chosen-ciphertext attack. In addition, a protocol failure of the type discussed in Exercise 12.2.4 applies to Rabin.
< previous page | page_291 | next page >


page_292

< previous page | page_292 | next page >

Page 292

A difficulty in Rabin decryption is that the correct plaintext must be identified from four candidates. One practical solution is to add structural requirements (e.g., replicated bits) to the messages in such a way that it is unlikely that more than one of the roots will possess the proper structure. This also thwarts the chosenciphertext attack to recover roots (since the oracle returns no useful information to the attacker in this case). However, the proof of equivalence to the factoring problem no longer holds for this modified scheme. Exercise 12.3.3 examines a Rabin-like scheme (on Blum integers n) that permits identification of the actual message.

Exercises

12.3.2 Let n = 551 in a (toy) Rabin encryption scheme.

(a) Encrypt the message m = 53 obtaining ciphertext c = m 2 mod n.

(b) Show the procedure for decrypting c, obtaining four candidates for m.

(c) Illustrate a chosen-ciphertext attack to factor n as follows. A random [image] is selected, say x = 53. The value c = x2 mod n is passed to the oracle for RABIN. What happens if the oracle returns 498? If the oracle returns 517, show that n can be factored.

12.3. This exercise outlines a scheme proposed by Williams [96, 97] for certain composite n which is similar to that of Rabin. Recovering plaintext from ciphertext is equivalent to factoring n, but the ambiguity in selecting the proper root is eliminated.

Let n = pq with p ≡ q ≡ 3 (mod 4), and define d = ((p − 1)(q − 1)/4+1)/2. Choose s so that [image]. The public key is (n, s) and d is the private key.

Encryption: Given message m with (m, n) = 1, calculate b 1 ∈ {0, 1} so that [image] [image] and calculate [image] The triple [image] is sent.

Decryption: Given the triple (c,b 1,b 2), calculate m 0 = ±cd mod n, where the sign is chosen so that [image] Then [image] is the recovered text.

(a) If [image] = 1 and c = x 2 mod n, show that cd ≡ ± x (mod n).

(b) Verify that the decryption procedure recovers m.

(c) Let p = 7, q = 11, and n = pq. Note that p ≡ q ≡ 3 (mod 4). Verify that s = 2 satisfies [image] = −1. Illustrate the encryption and decryption steps for the message m = 31.

The basic idea used in this scheme is that a distinguished square root (among the 4 possible) can be found in the case that n = pq with p ≡ q ≡ 3 (mod 4). Following the notation in Williams' paper, let [a, b] denote the residue class modulo n such that if y ∈ [a, b] then y ≡ a (mod p) and Y ≡b (mod q).
< previous page | page_292 | next page >


page_293

< previous page | page_293 | next page >

Page 293

(d) Show that [image]. Hence, if (c,p) = 1 then exactly one of ±c is a quadratic residue modulo p.

(e) If (y,n) = 1, show that the square roots of y 2 in [image] can be written [a,b], [−a, −b],[−a,b], and [a, −b], where a ∈ Qp and b ∈ Qq.

(f) If y ∈ [a,b] or y ∈ [−a,−b], then y is said to be a type 1 root of y 2; the other roots are said to be of type 2. Show that y is type 1 iff [image]; hence factoring is not necessary to determine type.

(g) If y 1 and y 2 are the distinct type 1 roots, then y 1 ≡ −y 2 (mod n) and exactly one of these roots is even.

An adversary with the ability to decrypt obtains a type 1 and a type 2 root of some number, and is then able to factor n.

(h) The adversary seeks an x with [image] and for which the decryption procedure can be applied to x 2. If every valid ciphertext can be decrypted, then x = s may be selected. The adversary decrypts (x 2,0,0), obtaining a type 1 root y. Verify that (y − x,n) is a nontrivial factor of n.

12.4 ElGamal

The presumed intractability of the discrete logarithm problem is the basis for encryption and signature schemes proposed by ElGamal in 1985 [29]. The first recognized digital signature algorithm, adopted by the US Government in 1994 as the Digital Signature Standard (DSS), is a variant of the ElGamal signature scheme.

ElGamal encryption requires a prime p and a generator α of [image]. Each user selects a random a, 1 ≤ a ≤ p − 2, as a private key, and the corresponding public key is (p,α,αa mod p). To send a message m, 0 ≤ m < p, using the public key, a random k, 1 ≤ k < p, is selected, and the pair (αk, m(αa)k) mod p is delivered. Possession of the private key a allows recovery of m, since α−ak can be computed and then m(αa)kα−ak mod p = m.

Example 12.4.1 (ElGamal encryption with toy parameters) Consider the case p = 13. We may select generator α = 2 of [image]. For the private key a, 1 ≤ a ≤ 13 − 2, suppose Alice chooses a = 6. Alice calculates αa mod p = 26 mod 13 = 12, and publishes (p,α,αa mod p) = (13, 2, 13) as her public key.

To send a message m = 9, Bob selects a random k, 1 ≤ k < p, perhaps k = 3. Using the public key, Bob delivers

(y,δ) = (αk, m(αa)k) mod p = (23,9(12)3) mod 13 = (8,4)

to Alice. On receipt, Alice may recover the message with her private key a = 6 by calculating
< previous page | page_293 | next page >


page_294

< previous page | page_294 | next page >

Page 294

and then m = δα−ak mod p = 4 · 12 mod 13 = 9. Note that key-generation in this example has given αa ≡ -1 (mod p), which is probably undesirable (see Exercise 12.4.5).

ElGamal has explicit randomization in its encryption, and a message m will map to different ciphertexts depending on the choice of the random k. This may be a desirable feature, making some attacks less feasible. Randomization added to the message in an RSA-based scheme has similar advantages. On the downside, ElGamal suffers from message expansion (since ciphertext consists of pairs of integers each as large as the message itself).

Consider the problem of interest to an adversary:

EL GAMAL: Given prime p, generator α of [image] and m(αa)k, find m.

Clearly, if there is a polynomial-time algorithm for the discrete log problem, then EL GAMAL may be solved efficiently; that is EL GAMAL ≤ DLP. The security is said to be based on discrete logs, although it is unknown if EL GAMAL is equivalent to DLP.

Note that the exchange of αa and αk is part of a Diffie-Hellman key agreement to obtain a session key αak; ElGamal then uses the session key to encrypt the message m by multiplication. An algorithm for the Diffie-Hellman problem (DHP) immediately solves the ElGamal problem. To see that DHP ≤ EL GAMAL, suppose we have a polynomial-time oracle for the ElGamal problem; that is, given an instance (p, α, αa, αk, mαak), the oracle returns m in polynomial time. In the Diffie-Hellman problem, we seek αak from (p, α, αa, αk). The oracle for EL GAMAL is given (p, α, αa, αk, 1), and m = α−ak is returned; inverting can be done in polynomial time, giving αak.

Signatures

The ElGamal signature scheme uses a hash function to map a message m of arbitrary length to a message digest x, which is then signed. Verification requires the message m as part of the process, and hence ElGamal is an example of a signature scheme with appendix.

Key generation for signatures is the same as for encryption. Suppose m ∈ {0, 1}* is to be signed. A known hash function H : {0, 1}* → {0, ..., p – 1} is used to obtain a message digest x = H(m). A random k, 1 ≤ k < p, is selected with (k, p − 1) = 1, and then r = αk mod p is calculated. The congruence x ≡ ar + ks (mod p − 1) is solved for s (using the private key a), and the pair (r, s) is the signature on m.

Verification from the public key uses the fact that αx ≡ αar+ks ≡ (αa)r(αk)s (mod p) for a valid signature. Given a purported signature (r, s) on m, x = H(m) is obtained from the known hash function. The signature is accepted if 1 ≤ r < p

< previous page | page_294 | next page >


page_295

< previous page | page_295 | next page >

Page 295

and the values of αx and (αa)r(αk)s agree (see Exercise 12.4.3 concerning the first condition).

An adversary wishing to forge a signature on a message m may calculate x = H(m) and r = αk for any k, but the congruence x ≡ ar + ks (mod p – 1) has unknowns a and s. However, it is possible to construct an x and signature (r,s) for which the verification condition αx ≡ (αa)r (αk)s holds. Choose integers j,k with 1 ≤ k < p and (k, p − 1) = 1, and let

r = αj(α)a)k mod p,
s = –rk-1), and
x = sj mod (p – 1).

To see that (r, s) is a valid signature on x, note that (αa)r r s ≡ αarαjsαaks ≡ αjs ≡ αx (mod p). The signature would be accepted if a message m which hashes to x can be found. In this case, the deception is known as an existential forgery, since the adversary has little control over the content of m.

A more interesting break of the ElGamal signature scheme would occur if an adversary could solve αx ≡ (αa)r r s (mod p) for the signature (r, s). Of course, if the private key a could be recovered (perhaps from a known signature), then the scheme is completely broken; however, recovering a from public information appears to be the discrete log problem. Solving for s in terms of r is also a discrete log problem, and solving for r in terms of s leads to a mixed exponential congruence in r for which no efficient method of solution is known.

For security, the modulus p must be sufficiently large so that the discrete log problem in [image] is infeasible. In 1996, Menezes et al. [63] noted that ''a 512 bit modulus p provides only marginal safety from concerted attack." A modulus of at least 768 bits was recommended, and 1024 bits or more should be used for longterm security. Unfortunately, the signatures are twice as long as the modulus, which may be of concern in applications such as smartcards.

The Digital Signature Standard (DSS), adopted in 1994, uses a variant of the ElGamal scheme. Although the modulus p is 512–1024 bits, a 320-bit signature is obtained on a 160-bit hash by working in a subgroup of [image]. A brief summary follows; see [63] for details on the Digital Signature Algorithm (DSA) and the Secure Hash Algorithm (SHA-1) used as the hashing function.

Key generation in DSA chooses a 160-bit prime q and a prime number p so that q | p − 1 and so that p has 512+64t bits for some 0 ≤ t ≤ 8 (for a ceiling of 1024 bits). Let β be a generator of the cyclic subgroup of order q in [image] (see Exercise 12.4.4). A private key a, 1 ≤ a < q, is randomly selected; the corresponding public key is (p, q, β, βa mod p).

To sign a message m, the 160-bit hash x for m is obtained. A random k, 1 ≤ k < q, is selected, and then the signature (r, s) is calculated by r = (βk mod

< previous page | page_295 | next page >


page_296

< previous page | page_296 | next page >

Page 296

p) mod q and s = (x+ar)k−1 mod q. Verification accepts (r, s) as a valid signature on m if the condition

holds (see Exercise 12.4.4).

Exercises

12.4.2 Illustrate the ElGamal signature scheme in this toy example. Suppose Alice chooses p = 17, generator α = 3 of [image] and private key a = 6; the corresponding public key is then (p, α, αa mod p) = (17, 3, 15). The (toy) hash function H is given by H(m) = m mod p.

(a) Find the hash value x and signature (r, s) on the message m = 26, assuming that Alice chooses k = 11 (so that (k, p − 1) = 1 is satisfied).

(b) Show the details of the verification of (r, s) on m. (Show clearly that the process does not require knowledge of the private key a.)

12.4.3 If the condition 1 ≤ r < p is not checked in the verification of an ElGamal signature, then a signature on a given message m′ can be forged provided a valid signature (r, s) on some digest x with (x, p − 1) = 1 is known. In this case, let x′ = H(m′) and u = x′ x−1 mod (p − 1). Define s′ = su mod (p − 1) and note that the system

r′ ≡ ru (mod p – 1)
r′ ≡ r (mod p)

has a solution r′ by the Chinese remainder theorem. Verify that (r′, s′) is a signature on m′ which will be accepted if the requirement that 1 ≤ r′ < p is omitted. (This exercise is adapted from [10]; see also [63, Note 11.66].)

12.4.4 This exercise examines two details in the Digital Signature Algorithm.

(a) Suppose that [image] satisfies β = g(p−1)/q mod p ≠ 1. Show that β has order q.

(b) If s ≠ 0, show that the verification condition holds.

12.4.5 In Example 12.4.1, key-generation yielded αa ≡ – 1 (mod p). Why is this a bad choice? (Hint: what is m(αa)k for each choice of k?)

12.4.6 (Weak generators) Suppose p ≡ 1 (mod 4) and the generator α of [image] satisfies α | p − 1. If computing logarithms in the subgroup G of order α in [image] is feasible, then an adversary can construct a signature (r, s) on a given message m as follows.

Let αa be Alice's public key, and let r be defined by p − 1 = αr.
< previous page | page_296 | next page >


page_297

< previous page | page_297 | next page >

Page 297

(a) Verify that αr is a generator of G. Hence, it is feasible to find z such that αrz ≡ (αa)r (mod p)

(b) Show that r(p–1)/2 ≡ –1 (mod p).

(c) Let [image]. Verify that (r,s) will be accepted as a signature on m.

This exercise is adapted from [10]; see also [63, Note 11.67].

12.4.7 Re-use of the random k in ElGamal could be disastrous.

(a) Suppose that the same k is used for encrypting messages m 1 and m 2. Show that an adversary in possession of the ciphertexts and m 1 ≠ 0 can efficiently recover m 2.

(b) Suppose that messages m 1 and m 2 are signed using the same k. Show that an adversary is likely to recover the private key.

12.5 Cryptographic protocols

A framework of procedures in which cryptographic primitives are applied is known as a protocol, defined informally in [25] as ''an algorithm for implementing a class of transactions (logical units of communication activity)." This short section introduces several protocol-related topics. Two "classical" problems are discussed: in the coin-tossing protocol, two parties (who do not trust each other) wish to flip a coin by telephone to resolve a dispute; the idea in zero-knowledge is to provide proof of possession of a secret without revealing anything about the secret itself. The need to clearly identify the assumptions in the protocol is illustrated by an active attack on the Diffie-Hellman key exchange. Even when the assumptions are clear, it may be unknown if they are in fact met in a specific case; an example of this type is given by the mental poker protocol (a fair game of poker without cards) of Shamir, Rivest, and Adleman [79], which was later shown to leak enough information to (partially) mark the cards.

When a security weakness is discovered, it is not always clear if the protocol or an underlying cryptographic function deserves the blame. Moore [64] writes: "The distinction seems to be that when a weakness is reported in a cryptosystem, the effect of which is to merely limit the scope of application or more clearly define the range of parameters that should be used for the algorithm, then the flaw discovered probably represents a protocol failure. However, if the effect is to leave the cryptosystem useless in any setting or to so severely restrict the possible range of parameters that the definition of a strong cryptofunction is infeasible, then the cryptosystem is actually 'broken'." Hence, the weakness in Exercise 12.2.4 is a protocol failure according to Moore. In Exercise 10.3.8, two apparently secure schemes were combined in such a way that encryption compromised authentication. It can be argued that this is a kind of protocol failure, although Moore's distinction would label this a broken (by design, actually) cryptosystem.
< previous page | page_297 | next page >


page_298

< previous page | page_298 | next page >

Page 298

Shamir's 3-pass protocol The difference between a protocol and a cryptographic function is illustrated by Shamir's 3-pass protocol, designed to achieve confidentiality dentiality with no advance exchange of keys. A symmetric-key encryption scheme is selected with the property that E k1 E k2 = E k2 E k1 for all keys k 1, k 2. Encryption can be viewed as adding a ''lock" to the envelope containing the message. The following protocol delivers a message m from A to B.

1. A and B pick random private keys kA and kB, respectively.

2. A adds her lock to the message, sending c 1 = EkA(m) to B.

3. B adds his lock, returning c 2 = EkB(c 1) = EkB EkA(m) to A.

4. A removes her lock, sending c 3 = DkA(c 2) to B. B decrypts c 3 to recover m.

Under suitable complexity assumptions on E, it appears that the isolated exchanges are secure from passive attacks. However, the protocol places additional (implicit) assumptions on the cipher. Consider the case that the cipher is the onetime pad, Ek(m) = k ⊕ m. A passive adversary obtains c 1 = kA ⊕ m, c 2 = kB ⊕ c 1, and c 3 = k B ⊕ m. Since c 1 ⊕ c 2 ⊕ c 3 = m, the transfer is totally insecure, even though an unconditionally secure cipher was employed. Exercise 12.5.1 suggests an alternative cipher for use in this protocol.

12.5.1 Diffie-Hellman key agreement

In the key agreement described on page 279, Alice and Bob exchange αa mod p and αb mod p over an unsecured channel, arriving at the shared secret αab. The scheme is secure against passive attacks, but not against active attacks.

If the keys are not authenticated, then in addition to impersonation, an active adversary may mount an intruder-in-the-middle attack which will go undetected by Alice and Bob. The adversary Eve shares the secret αab′ with Alice and αa′b with Bob, where Eve has selected a′ and b′. If Alice sends a message encrypted with a key derived from αab′, then Eve intercepts the message, decrypts it, re-encrypts with a key derived from αa′b, and passes it on to Bob. In this fashion, Eve obtains the entire plaintext correspondence.4

A trusted party or Certification Authority (CA) is sometimes used to provide authentication. After verifying the identity of an entity A, the CA forms a message containing identifying information (name, address, etc.), along with A's public key (perhaps αa in the present context). The message is signed by the CA, forming a "certificate" CA binding A's identity to her public key. Alice and Bob exchange

4See also Exercise 12.5.5 for a suggested protocol to force Eve to act nontransparently.
< previous page | page_298 | next page >


page_299

< previous page | page_299 | next page >

Page 299

their certificates CA and CB, which include αa and αb, respectively. On receipt of C A, Bob verifies the CA's signature, and is thus convinced that Alice's public key is αa. Similarly, Alice is convinced that Bob's public key is αb. Impersonation is still possible in the sense that an intruder may forward a certificate belonging to, say, Alice, but will not be able to form the shared secret αab.

Use of a CA in this fashion requires an expensive one-time creation of the certificate for each entity, but the verification of signatures does not require online support from the CA. The CA's public key must be authenticated, but secrecy is not required. On the downside, a CA providing services to a large number of clients becomes a very attractive target, and compromise of the CA may be catastrophic. Also, procedures are needed to distribute revocation information in the case that individual certificates or CA signatures become invalid.

Certificates are commonly used in secure purchases via internet. The authentication is typically one-way (the merchant delivers a certificate to the client), since client-side certificates are rare in this context. The client, perhaps using a web browser such as Netscape, maintains a list of Certification Authorities from which signatures will be accepted. There is a rather optimistic protocol assumption that the person on the client-side will actually examine the certificate. A valid certificate from http://www.delta.com offers convincing evidence that the public key for www.delta.com has been obtained; however, the person expecting to purchase plane tickets may be surprised to learn that Delta Air Lines is at the address www.delta-air.com.5

12.5.2 Zero-knowledge proofs

Alice (the prover) wishes to convince Bob (the verifier) that she possesses a secret s. One option open to Alice is to simply reveal the secret—of course, it is then no longer her secret. In the context of the password authentication scheme discussed in Section 10.3.2, an adversary obtains Alice's password, and can then impersonate Alice. Zero-knowledge protocols are designed to allow the prover to provide convincing evidence that she possesses the secret, without revealing anything that could be used by the verifier in subsequent demonstrations.

A rigorous discussion of zero-knowledge in interactive protocols involves nontrivial and somewhat subtle concepts. Readers interested in more formal or extensive treatments than presented in this brief overview should consult the references mentioned in Section 12.6 (where two non-mathematical introductions to zero-knowledge concepts are also noted).

As an illustration of (not necessarily zero-knowledge) proof systems, consider the problem of proving that a given v ∈ Jn is a quadratic nonresidue, where n is a product of large and secret primes. A prover in possession of the factorization of

5deltaComm remarks that 8000 visitors per day access www.delta.com looking for Delta Air. In 1999, modifications in their web pages were made to minimize the data-transfer cost of the unwelcomed accesses.
< previous page | page_299 | next page >


page_3

< previous page | page_3 | next page >

Page 3

In many cases, the information to be sent is transmitted by a sequence of zeros and ones. We call a 0 or a 1 a digit. A word is a sequence of digits. The length of a word is the number of digits in the word. Thus 0110101 is a word of length seven. A word is transmitted by sending its digits, one after the other, across a binary channel. The term ''binary" refers to the fact that only two digits, 0 and 1, are used. Each digit is transmitted mechanically, electrically, magnetically, or otherwise by one of two types of easily differentiated pulses.

A binary code is a set C of words. The code consisting of all words of length two is

C = {00, 10, 01, 11}.

A block code is a code having all its words of the same length; this number is called the length of a code. We will consider only block codes. So, for us, the term code will always mean a binary block code. The words that belong to a given code C 0, will be called codewords. We shall denote the number of codewords in a code C by |C|.

Exercises

1.2.1 List all words of length 3; of length 4; of length 5.

1.2.2 Find a formula for the total number of words of length n.

1.2.3 Let C be the code consisting of all words of length 6 having an even number of ones. List the codewords in C.

We also need to make certain basic assumptions about the channel. These assumptions will necessarily shape the theory that we formulate.

The first assumption is that a codeword of length n consisting of 0's and 1's is received as a word of length n consisting of 0's and 1's, although not necessarily the same as the word that was sent.

The second is that there is no difficulty identifying the beginning of the first word transmitted. Thus, if we are using codewords of length 3 and receive 011011001, we know that the words received are, in order, 011, 011, 001. This assumption means, again using length 3, that the channel cannot deliver 01101 to the receiver, because a digit has been lost here.

The final assumption is that the noise is scattered randomly as opposed to being in clumps called bursts. That is, the probability of any one digit being affected in transmission is the same as that of any other digit and is not influenced by errors made in neighboring digits. This is not a very realistic assumption for many types of noise such as lightning or scratches on compact discs. We shall eventually consider this type of noise.

In a perfect, or noiseless, channel, the digit sent, 0 or 1, is always the digit received. If all channels were perfect, there would be no need for coding theory. But fortunately (or unfortunately, perhaps) no channel is perfect; every channel is noisy. Some channels are less noisy, or more reliable, than others.
< previous page | page_3 | next page >


page_30

< previous page | page_30 | next page >

Page 30

Exercises

2.2.4 Construct examples in K 5 of each of the following rules

(a) u·(v + w) = u·v+u·w

(b) a(v·w) = (av)·w = v·(aw).

2.2.5 Prove that the two rules in Exercise 2.2.4 hold in Kn.

Vectors v and w are orthogonal if v·w = 0. The example above shows that v = 11001 and w = 01101 are orthogonal in K 5. For a given set S of vectors in Kn, we say a vector v is orthogonal to the set S if v·w = 0 for all w in S; that is, v is orthogonal to every vector in S. The set of all vectors orthogonal to S is denoted by S⊥ and is called the orthogonal complement of S.

In linear algebra it is shown that for any subset S of a vector space V, the orthogonal complement S⊥ is a subspace of V. For the vector space Kn, if C = 〈S〉, then we write C⊥ = S⊥ and call C⊥ the dual code of C.

Example 2.2.6 For S = {0100, 0101}, we compute the dual code C⊥ = S⊥. We must find all words v = (x, y, z, w) in K 4 such that both the equations

v·0100 = 0

v·0101 = 0

hold. Computing the scalar product we have

y = 0 and y+w = 0.

Thus y = w = 0 but x and z can be either 0 or 1. Writing down all such choices for v we get

C⊥ = S⊥ = {0000, 0010, 1000, 1010}.

Exercises

2.2.7 Find the dual code C⊥ for each of the codes C = 〈S〉 in Exercise 2.2.3.

2.2.8 Find an example of a nonzero word v such that v·v = 0. What can you say about the weight of such a word?

2.2.9 For any subset S of a vector space V, (S⊥)⊥ = 〈S〉. Use the example above to construct an example of this fact in K 4.

2.2.10 Prove that 〈S〉 [sube] (S⊥)⊥. (In fact (S⊥)⊥ = 〈S〉; for a linear code C, this means (C⊥)⊥ = C.)

2.3 Independence, basis, dimension

We review several important concepts from linear algebra and illustrate how to apply these concepts to linear codes. The main objective is to find an efficient way to describe a linear code without having to list all the codewords.
< previous page | page_30 | next page >


page_300

< previous page | page_300 | next page >

Page 300

n could offer irrefutable proof that [image] by exhibiting the factors. Considerably more than simply ''v is indeed a quadratic nonresidue" has been revealed to the verifier, however.

A more interesting proof system for the problem is given in [40]. In this context, "proof" is understood to mean "convincing evidence" rather than certainty.

1. The verifier B chooses a security parameter t > 0, and random bi ∈ {0, 1} and [image]. B then challenges the prover A with the values [image] mod n, 1 ≤ i ≤ t.

2. A determines (somehow) if each wi is a quadratic residue, and gives the response ci = 0 if wi is a quadratic residue and ci = 1 otherwise, 1 ≤ i ≤ t.

3. B checks if bi = ci for every i; if so, B accepts the proof (that v is a quadratic nonresidue).

If v is a quadratic nonresidue, then the challenge wi is a quadratic residue if and only if bi = 0. If t is sufficiently large, and both parties follow the protocol, then B will be convinced that v is a quadratic nonresidue. On the other hand, if v is in fact a quadratic residue, then every wi is a quadratic residue and the probability that bi = ci for every i is apparently 2−t, making it unlikely that B will accept a claim that [image].

If both parties follow the protocol, then B learns nothing beyond what he could have calculated without A (other than the fact that prover is indeed able to respond successfully to the challenges). The protocol is not zero-knowledge, however. The verifier is not forced to follow the protocol, and may obtain responses for wi of his choice; that is, the verifier is able to use the prover (provided the prover is not also cheating) to determine if elements of his choice (and not necessarily of the form in the protocol) are quadratic residues. A zero-knowledge version of this protocol in [40] essentially forces the verifier to play fair.

As mentioned above, one application of zero-knowledge concepts is in identification. The following protocol is built on a zero-knowledge proof that a known value v is a quadratic residue modulo n.

Protocol (Fiat-Shamir identification) A trusted center chooses an RSA-like modulus n = pq, keeping the factors private. The prover A obtains a secret [image] from the trusted center; v = s 2 mod n is A's public key. The prover convinces the verifier B of her possession of s by successfully completing the following 3-pass protocol t times, where t > 0 is a security parameter.

1. A chooses a random commitment r, 1 ≤ r < n, and sends the witness x = r 2 mod n to B.

2. B replies with a random challenge e ∈ {0, 1}.

3. A's response is y = rse mod n.

4. B verifies that y ≠ 0 and y 2 ≡ xv e (mod n).

B accepts the proof if all t rounds succeed.
< previous page | page_300 | next page >


page_301

< previous page | page_301 | next page >

Page 301

The verifier learns nothing about s: the response y = r is independent of s, and the random r in the response y = rs is unknown to B. In each round, an adversary impersonating A can cheat successfully if the challenge can be predicted: if e = 0 is expected, the witness and response are unchanged; if e = 1 is expected, then x = r2 v−1 is selected as the witness and y = r as the response. Since the challenges are selected at random, the probability of a successful prediction is only 1/2 for each round. Exercise 12.5.2 shows that such an imposter is likely to be detected. This inability to predict the challenges also thwarts a replay attack from an adversary in possession of a transcript from a valid protocol session.

12.5.3 Coin-tossing and mental poker

Two additional cryptographic protocols are considered in this section, giving a glimpse of the wide range of applications addressed by such protocols, and further emphasizing the need to identify protocol assumptions and verify that these requirements are met. Section 12.6 contains references to many other applications and attacks on such protocols.

Coin-tossing by telephone was suggested by Blum [11]. Alice and Bob have experienced a messy divorce, and wish to flip a coin by telephone to see who gets the kids. The predicament seems to be that neither party is willing to go first in declaring their call or in announcing the result of the toss. The basic idea in the following protocol is that Alice commits to ''heads" or "tails" and announces the commitment in a way that conceals the choice but binds her to the selection. Bob makes his guess at her choice, and then Alice reveals information which makes the commitment public. The security depends on the intractability of the quadratic residuosity problem.

Protocol (coin-tossing)

1. Alice selects n = pq for odd primes p ≠ q and a random x ∈ Jn, and announces n and x to Bob.

2. Bob responds with either "x ∈ Q n" or [image] (with 50% chance of being correct, assuming QRP is hard).

3. Alice reveals p and q. Bob verifies that p and q are indeed prime (and that x ∈ Jn); the correctness of Bob's response is determined by

If Bob fails to verify that p and q are prime, then Alice can cheat by forming n = p1p2p3 for primes pi, and choosing x with [image] = −1 and [image] = 1 (so that [image] = 1). Once Bob announces his guess, then Alice reveals the pair (p = p1p2, q = p3) or (p = p1, q = p2p3), depending on the outcome she desires (e.g., if she wants Bob to think that x is a quadratic residue, then she reveals the former).
< previous page | page_301 | next page >


page_302

< previous page | page_302 | next page >

Page 302

Mental poker Cryptographers are apparently quite interested in a fair game of poker without cards. In 1979, Shamir, Rivest, and Adleman suggested a protocol for dealing, published in The Mathematical Gardner in 1981. The basic idea is to use Shamir's 3-pass protocol. Alice and Bob agree on a set of messages mi, 1 ≤ i ≤ 52, identifying the 52 cards. Alice encrypts these messages, sending [image] (mi), 1 ≤ i ≤ 52, in random order. Bob selects five ciphertexts as Alice's hand, and returns them to her. He encrypts an additional five ciphertexts with [image], sends these to Alice, who removes her locks with [image] and returns the results as Bob's hand.

Encryption with RSA-like key-generation was suggested. Alice and Bob agree on a modulus n (prime or a product of distinct odd primes), and each selects a secret key-pair k = (e, d) with (e, ø(n)) = 1 and ed ≡ 1 (mod ø(n)), and then Ek(m) = me mod n and Dk(c) = cd mod n. The key-pairs are revealed at the end of the game.

Unfortunately, shortly after the scheme was proposed, Lipton showed that the suggested function fails to meet an implicit assumption in the protocol, that the cards cannot be marked [25]. The Jacobi symbol can be efficiently calculated, and the value is preserved by encryption; i.e., [image]. Unless the Jacobi symbol gives the same result for every mi, Bob can select cards with a specific Jacobi symbol to return to Alice, possibly offering an advantage. This particular attack may be defeated by choosing encodings mi which are all quadratic residues. The interested poker player may find references in Section 12.6 for suggested protocols and ways to cheat.

Exercises

12.5.1 In Shamir's 3-pass protocol, suppose that Ek(m) = mk mod p for suitable prime p.

(a) Verify that the encryption scheme meets the commutativity requirement for use in the protocol.

(b) How will kA and kB be selected? Find c 1, c 2, and c 3 in the exchange.

(c) Discuss the security.

12.5.2 This exercise concerns the Fiat-Shamir identification protocol.

(a) Show that if an impersonator successfully predicts a challenge e, then the witness x and response y will be accepted by the verification step.

(b) If the prediction turns out to be incorrect, explain why the impersonator is likely to be discovered.

12.5.3 (Proof of possession of a discrete log [20, 19]) Let p be prime and g an element of prime order q in [image]. Entity A has private key s, 1 ≤ s < q, and public key S = gs mod p. A proves knowledge of s to B in t > 0 rounds.
< previous page | page_302 | next page >


page_303

< previous page | page_303 | next page >

Page 303

1. A chooses random commitment x, 1 ≤ x < q, and sends the witness X = gx mod p to B.

2. B replies with a random challenge e ∈ {0, 1}.

3. A's reponse is y = x if e = 0, or y = sx−1 mod q if e = 1.

4. B checks that X = gy mod p if e = 0, or that S = Xy mod p if e = 1.

B accepts the proof if all t rounds succeed.

(a) If both parties follow the protocol, verify that B will accept.

(b) Indicate how an imposter can cheat, assuming that the challenges can be predicted (but s is unknown).

(c) Discuss the dilemma faced by an adversary in the case that a prediction turns out to be incorrect.

12.5.4 Consider the following proposed identification scheme. A certification authority binds Alice's identity to n = pq, where n is public and the primes p ≠ q are Alice's secret.

1. Bob challenges Alice with x, a random quadratic residue modulo n.

2. Alice's response is a square root y of x.

3. Bob verifies that y 2 ≡ x (mod n).

If several rounds are performed successfully, will Bob be convinced that Alice possesses the secret? Explain why this scheme is seriously flawed.

12.5.5 (Key agreement in the presence of active adversaries) Suppose that Alice and Bob can rely on voice recognition during key agreement. Rather than using the Diffie-Hellman shared secret αab directly, Alice and Bob establish a voice connection, and each reads a portion of αab. After verifying these pieces of αab, the remaining portion becomes the shared secret k.

(a) Assume that the adversary cannot efficiently recover k from αa, αb, and the leaked portions of αab. Will Alice and Bob be convinced that the adversary does not know k?

Rivest and Shamir [72] suggest a protocol change which forces an intruder-in-the-middle to act non-transparently (and thus possibly revealing her presence). Alice is to select a message m and send half of Ek(m), where k is the key that she has calculated (possibly shared with the intruder rather than with Bob); Bob will reply with half of the ciphertext for his selected message. Alice then transmits the remaining part of Ek(m), and Bob responds similarly.

(b) Assume that fragments of plaintext cannot be discovered from possession of only half of Ek(m). Discuss the options open to the adversary (in particular, describe what happens if she simply forwards the half of Ek(m) from Alice's first transmission).
< previous page | page_303 | next page >


page_304

< previous page | page_304 | next page >

Page 304

12.5.6 (Subliminal channel) Authentication schemes may permit an undetectable covert channel between parties. Simmons [82, 83] suggests the following scenario.

Prisoners' problem. Two accomplices in a crime have been arrested and are about to be locked in widely separated cells. Their only means of communication after they are locked up will be by way of messages conveyed for them by trustees—who are known to be agents of the warden. The warden is willing to allow the prisoners to exchange messages in the hope that he can deceive at least one of them into accepting as a genuine communication from the other either a fraudulent message created by the warden himself or else a modification by him of a genuine message. However, since he has every reason to suspect that the prisoners want to coordinate an escape plain, the warden will only permit the exchanges to occur if the information contained in the messages is completely open to him—and presumably innocuous. The prisoners, on the other hand, are willing to accept these conditions, i.e., to accept some risk of deception in order to be able to communicate at all, since they need to coordinate their plans. To do this they will have to deceive the warden by finding a way of communicating secretly in the exchanges, i.e., of establishing a ''subliminal channel" between them in full view of the warden, even though the messages themselves contain no secret (to the warden) information. Since they anticipate that the warden will try to deceive them by introducing fraudulent messages, they will only exchange messages if they are permitted to authenticate them.

The ElGamal signature scheme allows such a subliminal channel to be established between prisoners Alice and Bob. Key-generation for Alice remains unchanged: Alice chooses prime p, generator α of [image], and a private a, 1 ≤ a ≤ p – 2; her public key is (p, α, αa mod p).

Normally, Alice signs a message m by selecting a random k < p with (k, p – 1) = 1 and computing (r, s), where r = αk mod p and s is obtained from H (m) ≡ ar + ks (mod p – 1); then (m, r, s) is delivered to Bob through the warden. However, if Alice can arrange to share the secret a with Bob, then k may be used to transport the hidden message.6

(a) If Alice and Bob share a, identify conditions on s which will allow Bob to efficiently recover the subliminal message k from (m, r, s).

Given her current predicament, Alice may be willing to share her secret a with Bob (even though it means that Bob will be able to forge her signature), although

6 Although an ElGamal signature (r, s) may require 2log2 p bits, there are only log2 p bits of security; the remaining bits are available for the subliminal channel. However, since k is relatively prime to p – 1, only φ(p – 1) distinct messages k may be sent, and it may be infeasible to recover those for which xs ≡ H (m) – ar (mod p – 1) has many solutions x. Simmons [85, 84] observes that these "shortcomings" are avoided in the DSA, the "most hospitable setting for subliminal communications discovered to date."
< previous page | page_304 | next page >


page_305

< previous page | page_305 | next page >

Page 305

a method of securely delivering a is required. Young and Yung describe a SETUP (see Exercise 12.2.14) which may allow Alice to leak her private key to Bob, provided that Bob has a public key (p, α, αb mod p) known to Alice. Alice contaminates her signing process, producing (m 1, r 1, s 1) and (m 2, r 2, s 2) in such a way that possession of b permits recovery of a.

To leak a, Alice attempts to select a random k 1 with k 1, [image] mod p, and [image] mod p relatively prime to p – 1, where β−1 satisfies ββ−1 ≡ 1 (mod p – 1). Rather than choosing k 2 at random, Alice selects k 2 = β−1. Finally, let ri = [image] mod p and [image] mod (p – 1), i ∈ {1, 2}, as usual.

(b) Show that Bob can recover a by calculating [image] mod (p – 1)

(c) As an illustration, consider the toy example with p = 13 and α = 2. Verify that β and αβ–1 mod p are relatively prime to p – 1 if b = 5 and k 1 = 7. If b = 3, then show that Alice is unable to find k 1 satisfying the desired properties.

12.6 Notes

In addition to the schemes covered here, cryptographic methods based on elliptic curves have attracted considerable interest, in part because security is apparently maintained with shorter keys when compared with, say, RSA. The Elliptic Curve Digital Signature Algorithm (ECDSA, an elliptic curve analogue of the Digital Signature Algorithm) was accepted by the American National Standards Institute in 1999 (ANSI X9.62). Johnson and Menezes [44] discuss design decisions and related security, implementation, and interoperability issues for ECDSA; their paper also includes a short primer on elliptic curves. Koblitz [50] and Stinson [86] contain introductions to elliptic curves and applications to cryptography; more advanced texts on elliptic curve methods include Blake, Seroussi, and Smart [8] and Menezes [62].

Public-key methods were known to British cryptographers in the early 1970s, according to technical reports released by the Communications-Electronics Security Group (CESG) in December 1997 [31]. Known as ''non-secret encryption" within CESG [30], an RSA-like scheme appears in Cocks [21] and the ideas of the Diffie-Hellman key agreement are given in Williamson [100].7

7 CESG remarks "It is interesting to note that although many different ideas for public-key systems have been proposed, the two which have stood the test of time were the first two to be discovered. It is also interesting that the order of discovery in academia was the opposite to the order of discovery at CESG." Williamson [100] offers a few cautionary words in his introduction: "One of the reasons for the delay in writing this is that I find myself in an embarrassing position; having written [99], I have come to doubt the whole theory of non-secret encryption. The trouble is that I have no proof that the method of [99] is genuinely secure, in other words that it has a guaranteed work-factor involved in breaking it."
< previous page | page_305 | next page >


page_306

< previous page | page_306 | next page >

Page 306

Well-known applications that combine public-key and symmetric-key methods to provide authentication and confidentiality include Pretty Good Privacy (PGP) [37, 104] and (at the systems level) Sun Microsystems' remote procedure call (RPC) mechanism [88, 70]. It is interesting to note that a portion of the security in Sun's scheme rested on exponentiation modulo a 192-bit prime, considered insecure even in 1991 [54]. Vulnerability assessments must consider the larger framework of network security, which involves much more than cryptographic primitives. Indeed, security problems often do not directly involve cryptography. The 1990s saw a flood of security alerts and system compromises due to buffer overruns (caused by contrived messages which exceed implicit length assumptions) and shell escapes (privileged processing triggered by messages containing unexpected characters), some of which were in the very programs implementing security mechanisms.

The material on zero-knowledge is adapted from Goldwasser, Micali, and Rackoff [40] and Menezes, van Oorshot, and Vanstone [63]; see also Chapter 13 of Stinson [86]. Brassard and Crépeau [18] sort out various notions of zero-knowledge.

Interesting (or at least amusing) non-mathematical introductions to zero-knowledge have been presented at CRYPTO rump sessions. Quisquater, Guillou, and Berson [68] explore ''The Strange Cave of Ali Baba," which has an entry forking to passages with apparent dead ends. An interactive test is devised so that a claimant can provide convincing proof of possession of magic words which open a corridor between the dead ends, without revealing the secret itself. The claimant, who has entered the cave alone some time earlier, is asked to return using a passage selected by a witness placed at the fork. The process is repeated until the witness is convinced that the claimant possesses the magic words.8

In the CRYPTO '98 rump session [65], zero-knowledge was introduced with the help of Where's Waldo [41], a game where a certain figure (Waldo) is to be located within a complicated illustration. Alice wishes to prove to Bob that she has found Waldo, but does not want to reveal the solution. She could use her scissors to carefully remove Waldo from the background, but perhaps Bob will not be convinced, accusing her of using an image from some other source. Alice's solution is to use an opaque sheet of paper, twice the size of the illustration, with a small window in which to isolate Waldo.

Overviews on protocols (and attacks) for applications ranging from secret sharing and subliminal channels to anonymous digital cash and voting schemes appear in Salomaa [75], Seberry and Pieprzyk [77], and Schneier [76]. Simmons [84] gives a survey of subliminal channels, beginning with a historical introduction concerning verification proposals for the Strategic Arms Limitation Treaty (SALT II). Suggested protocols and ways to cheat at mental poker may be found in Fortune and Merritt [33] and Coppersmith [22] and the references therein.

8 The proof could have been provided in one step by asking the claimant to complete the loop beginning at the fork, rather spoiling the story (and the analogy with mathematical zero-knowledge).
< previous page | page_306 | next page >


page_307

< previous page | page_307 | next page >

Page 307

Appendix A
The Euclidean Algorithm

The greatest common divisor (or gcd) of two polynomials f(x), g(x) ∈ K[x] is the polynomial d(x) ∈ K[x] of largest degree such that f(x) = q1(x)d(x) and g(x) = q2(x)d(x); we will denote this by gcd (f(x), g(x)) = d(x).

Example A.1 We find the greatest common divisor of f(x) and g(x) assuming that we know the factorization of f(x) and g(x)) into irreducible polynomials where f(x) = 1+x 2+x 3+x 6+x 7+x 8 = (1+x)(1+x+x 3)(1+x 4) and g(x) = 1+x 3+x 5+x 6 = (1+x)(1+x 2)(1+x+x 3). The polynomial of highest degree which is a common factor of both f(x) and g(x) is 1+x+x 3. Thus

gcd(f(x), g(x) = 1+x+x 3.

Factoring f(x) and g(x), then hunting for the common factor of highest degree, is not an efficient way to find the greatest common divisor. Below we give a famous algorithm for accomplishing this task more readily.

Euclidean Algorithm Given f(x), g(x) ∈ K[x] with degree f(x) ≥ degree g(x) and g(x) ≠ 0

1. (Initialize) r 0(x) = f(x), r 1(x) = g(x), i = 1.

2. While ri(x) > 0, divide ri(x) into r i–1(x) and let r i+1(x) be the remainder. That is r i+1(x) = r i–1(x) mod r i(x). Increment i and repeat.

3. r i(x) = 0. Then gcd (f(x), g(x)) = r i–1(x).

Note that this algorithm must stop, after a finite number of steps, since for each i > 1, the degree of the remainder r i+1(x) is less than the degree of the remainder r i(x).

We can modify this algorithm to produce polynomial t i(x), s i(x) ∈ K[x] such that

t i(x)f(x)+s i(x)g(x) = r i(x) for i = 0,1,...
< previous page | page_307 | next page >


page_308

< previous page | page_308 | next page >

Page 308

Define

t 0(x) = 1 t 1(x) = 0
s 0(x) = 0 s 1(x) = 1

Assuming that r i–1(x) = q i(x)r i(x)+r i+1(x) (using the Division Algorithm) define

Then

Since we are working over the binary field we can ignore the minus signs.

Example A.2 We use the Euclidean Algorithm to find the greatest common divisor of the polynomials

f(x) = x 2+x 3+x 6+x 7
g(x) = 1+x 3+x 4+x 5.

The computations proceed as follows. Set i = 0, r 0(x) = f(x) and r 1(x) = g(x). Dividing r 1(x) into r 0(x) yields

x 2+x 3+x 6+x 7 = (1+x 3+x 4+x 5)(1+x 2)+(1+x 4).

Thus r 2(x) = 1+x 4 and q 2(x) = 1+x 2. Dividing r 2(x) into r 1(x) yields

1+x 3+x 4+x 5 = (1+x 4)(1+x)+(x+x 3).

Thus r 3(x) = x+x 3 and q 3(x) = 1+x. Next,

1+x 4 = (x+x 3)(x)+(1+x 2).

Thus r 4(x) = 1+x 2 and q 4(x) = x. Next

x+x 3 = (1+x 2)(x)+0,

so r 5(x) = 0. Since the last nonzero remainder is r 4(x) = 1+x 2, r 4(x) is the required common divisor of f(x) and g(x):

1+x 2 = gcd(1+x 3+x 4+x 5,x 2+x 6+x 7).
< previous page | page_308 | next page >


page_309

< previous page | page_309 | next page >

Page 309

In this example, we could also compute t i(x) and s i(x) using the quotients q i(x) computed in each iteration of (see table below). We claim that for each i, i = 0,1,2,3,4

r i(x) = t i(x)f(x)+s i(x)g(x).

It is obviously true for i = 0,1, and i = 2 since r 0(x)+q 1(x)r 1(x) = r 2(x). For i = 3 we have

Summarizing, we get:

i | t i(x) | s i(x) | r i(x)
0 | 1 | 0 | f(x)
1 | 0 | 1 | g(x)
2 | 1 | 1+x 2 | 1+x 4
3 | 1+x | x+x 2+x 3 | x+x 3
4 | 1+x+x 2 | 1+x 3+x 4 | 1+x 2
 | – | – | 0

Using induction one can prove the following.

Theorem A.3 If gcd(f(x), g(x)) = d(x) then there exist polynomials t(x) and s(x) in K[x] such that

t(x)f(x)+s(x)g(x) = d(x).

Exercises

A.4 Find the greatest common divisor of each of the following pairs of polynomials.

(a) f(x) = 1+x+x 5+x 6+x 7,g(x) = 1+x+x 3+x 5

(b) f(x) = 1+x 2+x 3+x 7,g(x) = 1+x+x 3

(c) f(x) = 1+x+x 4+x 5+x 8+x 9,g(x) = 1+x 2+x 3+x 7

(d) f(x) = 1+x+x 2+x 3+x 4,g(x) = x+x 3+x 4

A.5 Find gcd(f(x), g(x)) for f(x) = 1+x 9 and g(x) as given in each part.

(a) g(x) = x+x 2+x 4+x 5+x 7+x 8

(b) g(x) = x 3+x 6

(c) g(x) = 1+x+x 2+x 4+x 5+x 7+x 8
< previous page | page_309 | next page >


page_31

< previous page | page_31 | next page >

Page 31

A set S = {v 1, v 2,..., vk} of vectors is linearly dependent if there are scalars a 1, a 2,...,ak not all zero such that

a 1 v 1 + a 2 v 2 +...+ ak vk = 0.

Otherwise the set S is linearly independent.

The test for linear independence, then, is to form the vector equation above using arbitrary scalars. If this question forces all the scalars a 1, a 2,...,ak to be 0, then the set S is linearly independent. If at least one ai can be chosen to be nonzero then S is linearly dependent.

Example 2.3.1 We test S = {1001, 1101, 1011} for linear independence. Let a, b, and c be scalars (digits) such that

a(1001) + b(1101) + c(1011) = 0000.

Equating components on both sides yields the scalar equations

a + b + c = 0, b = 0, c = 0, a + b + c = 0.

These equations force a = b = c = 0. Therefore S is a linearly independent set of words in K 4.

Example 2.3.2 We test S = {110, 011, 101, 111} for linear independence. Consider

a(110)+b(011)+c(101)+d(111)=000.

This yields the system of scalar equations

a+c+d = 0

a+b+d = 0

b+c+d = 0.

Adding these three equations gives d = 0. Now we have a+c = 0, a+b = 0, b+c = 0. Thus we can choose a=b=c =1. Therefore S is a linearly dependent set.

In linear algebra it is shown that any set of vectors S ≠ {0} contains a largest linearly independent subset. The next example shows how such a subset may be found.

Example 2.3.3 Let S = {110, 011, 101, 111}. The last example shows that S is linearly dependent. In fact, we found that

1(110)+1(011)+1(101)+0(111) = 000,
< previous page | page_31 | next page >


page_310

< previous page | page_310 | next page >
Page 310 (d) g(x) = 1+x 3+x 6 (e) g(x) = x+x 2+x 3+x 4+x 5+x 6+x 7+x 8 A.6 Find gcd(f(x), g(x) for f(x) = 1+x 15 and g(x) = x+x 2+x 4+x 8. A.7 Find gcd(f(x), g(x) for f(x) = 1+x 23 and g(x) = x + x 2+x 3+x 4+x 6+x 8+x 9+x 12+x 13+x 16+x 18.
< previous page | page_310 | next page >


page_311

< previous page | page_311 | next page >

Page 311

Appendix B
Factorization of 1 + x n

The factorization of 1+xn into irreducible polynomials for 1 ≤ n ≤ 31, n odd.

n | Factorization
1 | 1+x
3 | (1+x)(1+x + x 2)
5 | (1+x)(1+x + x 2+x 3+x 4)
7 | (1+x)(1+x + x 3)(1+x 2+x 3)
9 | (1+x)(1+x + x 2)(1+x 3+x 6)
11 | (1+x)(1+x+...+x 10)
13 | (1+x)(1+x+...+x 12)
15 | (1+x)(1+x + x 2)(1+x+x 2+x 3+x 4)(1+x+x4)(1+x3+x4)
17 | (1+x)(1+x + x 2+x 4+x 6+x 7+x 8)(1+x3+x4+x5+x8)
19 | (1+x)(1+x + x 2+...+x 18)
21 | (1+x)(1+x + x 2)(1+x 2+x 3)(1+x+x 3) (1+x 2+x 4+x 5+x 6)(1+x + x 2+x 4+x 6)
23 | (1+x)(1+x + x 5+x 6+x 7+x 9+x 11)(1+x2+x4+x 5+x 6+x 10+x 11)
25 | (1+x)(1+x + x 2+x 3+x 4)(1+x5+x10+x15+x20)
27 | (1+x)(1+x + x 2)(1+x 3+x 6)(1+x 9+x 18)
29 | (1+x)(1+x+...+x 28)
31 | (1+x)(1+x 2+x 5)(1+x3+x5)(1+x+x2+x 3+x 5) (1+x + x 2+x 4+x 5)(1+x + x 3+x 4+x 5)(1+x 2+x 3+x 4+x 5)
< previous page | page_311 | next page >


page_312

< previous page | page_312 | next page >
Page 312 This page intentionally left blank.
< previous page | page_312 | next page >


page_313

< previous page | page_313 | next page >

Page 313

Appendix C
Example of Compact Disc Encoding

Since it would take too much space and time to do an example of Compact Disc encoding (see section 7.3), let us scale it down to something reasonable to do by hand. Consider the Reed-Solomon code C over GF(24) with generator g(x) = (1+x)(β+x)(β2+x)(β3+x) = β6+β0 x+β4 x 2+β12 x 3+x 4. This is a (15, 11, 5) code, which we could shorten to an (8, 4, 5) code C 1 or a (12, 8, 5) code C 2. These could be delay interleaved by two columns and to a depth of 8.

A message stream such as m would first be encoded to a codeword c in the code C 1 by using some generating matrix (see Table C.1).

Table C.1: Message stream and first encoding.
< previous page | page_313 | next page >


page_314

< previous page | page_314 | next page >
Page 314 To delay interleave these codewords in C 1, they should be viewed as The columns of this array are then viewed as messages and encoded to codewords in C 2 with each row of the following array a codeword:
< previous page | page_314 | next page >


page_315

< previous page | page_315 | next page >

Page 315

In binary, these codewords would be:

This could be modulated from 4-bit strings to 6-bit strings (say, having the property that at least 1 and at most 4 zeros occur between ones) by using the following table look-up:

Between 6-bit words, a bit is added (the nor of the two neighboring end bits) to maintain this property. The original message stream m (see Table C.1) would then finally appear as:
< previous page | page_315 | next page >


page_316

< previous page | page_316 | next page >
Page 316
< previous page | page_316 | next page >


page_317

< previous page | page_317 | next page >

Page 317

Appendix D
Solutions to Selected Exercises

1 Introduction to Coding Theory

1.2.1 (a) 000, 010, 100, 110, 001, 011, 101, 111 (b) 0000, 0100, 1000, 1100, 0001, 0101, 1001, 1101 0010, 0110, 1010, 1110, 0011, 0111, 1011, 1111

1.2.2 2n

1.2.4 Such a channel can be converted into a perfect channel by replacing each 1 with a 0 and each 0 with a 1.

1.2.5 Replace each 0 with a 1 and each 1 with a 0.

1.2.6 Nothing can be deduced about the codeword sent from the word received.

1.3.4 001

1.3.5 C = {0000, 0011, 0101, 0110, 1001, 1010, 1100, 1111} (a) Yes. (b) 0101, 1001, 1100, 1111. (c) No. Each word of length 4 which does not belong to C has 4 different closest codewords.

1.3.7 8 1.3.8 16, 32, 2n–1 1.4.1 1, 3/4, 1/3

1.6.2 (a) p 3(1–p)5 = 2.2 × 10−8, (b) p 7 = .81, (c) (1–p)5 = 2.4 × 10−8, (d) p 5 = .86, (e) p 4(1–p)3 = 2.4 × 10−5, (f) (1–p)5 = 2.4 × 10−8, (g) (1– p)6 7.3 × 10−10

1.6.5 0001110 1.6.6 101101101 1.6.7 00011 1.6.8 100110 1.6.9 110101 or 101000

1.6.10 (a) φp(v 1, w) ≤ φp(v 2, w) iff d 1 ≤ d 2 (b) φp(v, w) = (1/2)n for any w and any v.

1.9.5 If 000, 001, 010, or 011 is received then IMLD decides that 001 was sent. In all other cases IMLD incorrectly decides that 101 was sent.

1.9.6 000 is decoded as 000. 001, 011 and 101 are decoded as 001. 110 and 111 are decoded as 110. 010 and 100 require retransmission.

1.9.7 * in the following table indicates that retransmission is required.
< previous page | page_317 | next page >


page_318

< previous page | page_318 | next page >

Page 318

Received Word | Decoded to | Received Word | Decoded to
000 | * | 000 | 000
001 | * | 001 | 001
(a) 010 | 011 | (b) 010 | 010
011 | 011 | 011 | 011
100 | 101 | 100 | 000
101 | 101 | 101 | 001
110 | 111 | 110 | 010
111 | 111 | 111 | 011

1.10.2 (a) L(001) = {000, 001, 010, 011}. Therefore θ p(C, 001) = p 3 + 2p 2(1 – p) + p(1 – p)2. (b) L(101) = {100, 101, 110, 111}. Therefore θp(C, 101) = p 3 + 2p 2(1 – p) + p(1 – p)2.

1.10.4 (a) θ p(C, 110) = p 3 + p 2(1 – p) (b) To decode to 000 only 000 can be received, so θ p(110, 000) = p(1 – p)2.

1.10.5 (a) θ p(C, 101) = p 3 + p 2 (1 – p) for all v ∈ C. (b)θ p(C,v) = p3 + p2(1 – p) for all v ∈ C. (c)θp(C, 0000) = p4 + 3p3 (1 – p), θp (C, 0001) = p4 + 3p3 (1 – p) and θp(C, 1110) = p 4 + 4p 3 (1 – p). (e) θp(C, 00000) = θp(C, 11111) = p 5 + 5p 4 (1 – p) + 10p 3 (1 – p)2. (g)θp(C,v) = p 5 + 3p 4 (1 – p) for all v ∈ C. (h)θp (C,v) = p 6 + 6p 5 (1 – p) + 9p 4 (1 – p)2 for all v ∈ C.

1.11.2 (a) No (b) Yes (c) No

1.11.3 (a) (i) No (ii) Yes (iii) No (b) (i) Yes (ii) Yes (iii) No 1.11.4 None

1.11.7 (a) 001, 011, 101, 111 (c) K 4\{0000, 0001, 1110, 1111} (e) K 5\{00000, 11111} (h) K 6\{000000, 101010, 010101, 111111}

1.11.12 (a) 1 (b) 1 (c) 1 (d) 2 (e) 5 (f) 3 (g) 2 (h) 3

1.11.13 2 1.11.18 K 3\{000, 011, 101, 110}

1.11.19 (a) None (d) 1000, 0100, 0010 and 0001 (e) all error patterns of weight 1, 2, 3 or 4 (h) all error patterns of weight 1 or 2

1.12.12 (a) (i) 000, 001 (ii) 000 (c) (i) 0000, 0010, 0100, 1000 (ii) 0000 (f) (i) 00000, 10000, 01000, 00100, 00010, 00001 (ii) 00000, 10000, 01000, 00100, 00010, 00001 (g) (i) 00000, 01000, 00100, 00010 (ii) 00000

2 Linear Codes

2.1.1 (a) and (c) are not linear codes; the rest are linear codes.

2.2.3 (a) 〈S〉 = {000, 010, 011, 111, 001, 101, 100, 110} (b) 〈S〉 = {0000, 1010, 0101, 1111} (d) 〈S〉 = K 4

2.2.7 (a) C⊥ = {000} (b) C⊥ = {0000, 1010, 0101, 1111}
< previous page | page_318 | next page >


page_319

< previous page | page_319 | next page >

Page 319

(c) C⊥ = {0000, 1111}

2.3.4 (a) linearly independent (b) {101, 011, 010} (e) linearly independent (f) {1100, 1010, 1001} (i) {10101010, 01010101}

2.3.7 (a) B = {100, 010, 001}, B⊥ = φ (b) B = {1010, 0101}, B⊥ = B (c) B = {0101, 1010, 1100}, B⊥ = {1111} (e) B = {11000, 01111, 11110, 01010}, B⊥ = {11111}

2.3.8 (a) dim C = 3, dim C⊥ = 0 (b) dim C = 2, dim C⊥ = 2 (c) dim C = 3, dim C⊥ = 1 (e) dim C = 4, dim C⊥ = 1 (f) dim C = 3, dim C⊥ = 2

2.3.16 (a) dim C = 4 (b) |C| = 16 2.3.17 |C| = 32

2.5.3 (a) {100, 010, 001} (c) {1001, 0101, 0011} (e) {100001, 01001, 00101, 00011} (g) {101, 0101, 0011}

2.5.6 (a) {010, 011, 111} (c) {0101, 1010, 1100} (e) {11000, 01111, 11110, 01010} (g) {0110, 1010, 0011}

2.5.10 (a) ø (b) {1010, 0101} (e) {11111} (h) {101000, 110110, 000101}

2.5.12 (a) B = {111000, 000111} (b) B = {1000110, 0100011, 0010111, 0001101} (c) B = {1000001, 0100001, 0010001, 0001001, 0000101, 0000011} (f) B = {001000, 000100, 000010, 000001}
< previous page | page_319 | next page >


page_32

< previous page | page_32 | next page >

Page 32

so we can solve for 101 as a linear combination of the other words in S:

101 = 1(110) + 1(011) + 0(111).

In the dependent set S, if we take the words in the order given, we come to 101 as the first word which is dependent on, that is, is a linear combination of, the preceding words 110 and 011 in S. Discarding this word, we obtain a new set S′ = {110, 011, 111}. Now S′ can be tested for linear independence. If S′ is linearly dependent, we discard the first word which is a linear combination of the preceding words, thus obtaining a new set S″. This process may be repeated until we find a new set which is linearly independent; such a set is always a largest linearly independent subset of the given set S. In the present example, this set is S′.

Exercises

2.3.4 Test each of the following sets for linear independence. If the set is linearly dependent, extract from S a largest linearly independent subset.

(a) S = {1101, 1110, 1011}

(b) S = {101, 011, 110, 010}

(c) S = {1101, 0111, 1100, 0011}

(d) S = {1000, 0100, 0010, 0001}

(e) S = {1000, 1100, 1110, 1111}

(f) S = {1100, 1010, 1001, 0101}

(g) S = {0110, 1010, 1100, 0011, 1111}

(h) S = {111000, 000111, 101010, 010101}

(i) S = {00000000, 10101010, 01010101, 11111111}

In Exercise 2.3.4(i) S is found to be a linearly dependent set. Note that S contains the zero word. It is always true that any set of vectors containing the zero vector is linearly dependent.

A nonempty subset B of vectors from a vector space V is a basis for V if both:

1. B spans V (that is, 〈B〉 = V), and

2. B is a linearly independent set.

Note that any linearly independent set B is automatically a basis for 〈B〉. Also since any linearly dependent set S of vectors that contains a non-zero word always contains a largest independent subset B, we can extract from S a basis B for 〈S〉. If S = {0} then we say that the basis of S is the empty set, [image].

Example 2.3.5 Let S = {1001, 1101, 1011}. In Example 2.3.1 we found that S is linearly independent. Therefore S is a basis for the code C = 〈S〉 = {0000, 1001, 1101, 1011, 0100, 0010, 0110, 1111} which is a subspace of K 4.

Example 2.3.6 Let S = {110, 011, 101, 111}. In Example 2.3.2 we found that S is linearly dependent. But in Example 2.3.3 we extracted a maximal linearly independent subset B = S′ = {110, 011, 111} of S. Hence B is a basis for the code C = 〈S〉.
< previous page | page_32 | next page >


page_320

< previous page | page_320 | next page >

Page 320

2.6.10 (a) (i) 10011 (ii) 01010 (iii) 11100

2.6.11 10110, 01011, 01110, 00101, 01011, 10011, 01011

2.6.12 (a) 1001100, 0001011, 1110100, 1111111 (b) 0001100, 0001011, 1110101, 1111001

2.6.13 (2.6.6) (a) |C| = 8, R = ½ (b) |C| = 8, R = 1/3 (c) |C| = 4, R = 1/5 (2.6.7) (a) |C| = 16, R = ½ (b) |C| = 16, R = ½ (c) |C| = 8, R = 1/3 (d) |C| = 8, R = 3/5 (f) |C| = 8, R = 1/3 (g) |C| = 16, R = 4/7

2.7.10 C⊥ consists of the 16 words of even weight in K 5.

2.7.11 (a) dim C = t, dim C⊥ = 2t – t – 1, |C| = 2t |C⊥| = 22t–1–, R = t /(2t– 1) (b) dim C = 11, dim C⊥ = 12, |C| = 211 = 2048, |C⊥| = 212 = 4096, R = 11/23 (c) dim C = 8, dim C⊥ = 7, |C| = 28 = 256, |C⊥| = 27 = 128, R = 8/15

2.8.4 (a) 1111100 (b) 1011000

2.8.10 (a) C′ = {00000, 11100, 10101, 01001}

2.8.14 (a) Yes (b) Yes (c) No 2.9.4 (a) 4 (b) 4 (c) 4

2.10.6 (a) C, C + 1000, C + 0011 C + 0011 (b) C, C + 1000, C + 0100, C + 0001

2.10.7 (a) C, C + 100000, C + 010000, C + 001000, C + 000100, C + 000010, C + 000001, C + 001001 (d) C, C + 100000 (f) C, C + 1000, C + 0100, C + 0010, C + 0001, C + 1100, C + 1010, C + 1001

2.10.8 (a) C, C + 1000, C + 0100, C + 0001 (b) C, C + 1000000, C + 0100000,
< previous page | page_320 | next page >


page_321

< previous page | page_321 | next page >

Page 321

C + 0010000, C + 0001000, C + 0000100, C + 00000010, C + 0000001 (c) C, C + 000100, C + 010000, C + 001100, C + 100000, C + 100100, C + 110000, C + 110100

2.11.2 (a) 010011 (b) 101001 (c) 001111 (d) 010011 (e) 110101 (f) 001111.

2.11.8 (a)

2.11.9 (a) 2.11.10 (b)

2.11.19 (a) (i) 1100 (ii) 1001 (iii) 0101 (c) (i) 001110 (ii) 001110 (iii) 011011

2.11.21 (a)

2.12.2 (2.10.6) (a) θp(C) = p 4 + p 3 (1 – p) (c) θp(C) = p 5 + 3p 4 (1 – p) (2.10.7) (a) θp(C) = p 6 + 6p 5 (1 – p) (b)θp(C) = p 6 + 6p 5(1 – p) + 9p 4 (1 – p)2 (2.10.8) (a) θp(C) = p 4 + 2p 3 (1 – p) (b) θp(C) = p 7 + 7p 6 (1 – p)

3 Perfect and Related Codes

3.1.5 (a) 24 (b) 24 (c) 24 (e) 28 (f) 4096
< previous page | page_321 | next page >


page_322

< previous page | page_322 | next page >

Page 322

3.1.18 (a) (8, 6, 3), No 16 ≤ |C| ≤ 16 (d) (15, 6, 3), Yes 2048

3.1.19 (a) 64 ≤ |C| ≤ 256 (b) 2048 ≤ |C| ≤ 2048 (c) 128 ≤ |C| ≤ 128 (d) 256 ≤ |C| ≤ 256 (e) 32 ≤ |C ≤ 256 (f) 16 ≤ |C| ≤ 32

3.1.20 No

3.3.4

3.4.7 (a) 696 (b) 17 (c) 17

3.6.5 (a) 100000001001, 000000000000 (b) 000000100000, 001000010000 (c) 000000100000, 000000010000 (d) ask for retransmission (e) 011000000000, 00000000100 (g) 000000000000, 001010000000

3.6.6 (a) 010010000000, 000000000000 (b) 00000000000, 001000110000 (c) 001000000000, 100000000000 (d) 000000000101, 000000000001 (e) 000100000000, 000110000000 (f) 000001000000, 000000001000

3.7.3 (a) 111111100000, 10101111011 (b) 100000000000, 11011100010 (c) 000101011001, 11100000000 (d) 011000001001, 011011011011

3.7.7 253

3.8.5

3.8.10 (a) 0101 1010 (b) 0110 0110 (c) ask to retransmit (d) 1100 1100

3.9.6

(a) w 3 = (2, –2, 2, –2, –2, –6, –2, 2) m = (0101)

(b) w 3 = (2, –2, –2, –6, –2, 2, 2, 2) m = (0110)

(c) w 3 = (–4, –4, 0, 0, 0, 0, –4, 4) m = ?

(d) w 3 = (2, 2, 6, –2, –2, –2, 2, 2) m = (1010)
< previous page | page_322 | next page >


page_323

< previous page | page_323 | next page >

Page 323

4 Cyclic Linear Codes

4.1.10 (a) q(x) = x 3,r(x) = x 3

4.1.13 (a) {0+x 2,x, x + x 2} (c) {0,x 3,1 + x + x 2}

4.2.22 (a) g(x) = 1 (e) g(x) = 1+x

5 BCH Codes

5.1.17 β, β2, β4, β7, β8, β11, β13, β14

5.2.7

element | minimal polynomial
0 | x
1 | 1 + x
β, β2, β4 | 1 + x + x 3
β, β6, β5 | 1 + x 2 + x 3

5.2.8

element | minimal polynomial
0 | x
1 | 1 + x
β5, β10 | 1 + x + x 2
β7, β14, β13, β11 | 1 + x + x 4
β, β2, β4, β8 | 1 + x 3 + x 4
β3, β6, β9, β12 | 1 + x + x 2 + x 3 + x 4

5.5.9 (a) Ask for retransmission (b) 10 (c) 5 and 8 (d) 6 and 11 (e) Ask for retransmission (f) Ask for retransmission (g) 0 and 13 (h) Codeword

6 Reed-Solomon Codes

6.1.6 (a) 215 (b) g(x) = β+β3 x + x 2 (c) (i) βββ6β6000 (d) gk(x) = (1 + x)(β + x)(β2 + x)(β4 + x)

6.1.7 (a) 244 (b) g(x) = β10 + β3 x + β6x2 + β13 x 3 + x 4
(c)(i) β10β3β6β13 100000β2β10β13β5β7
(d) gk = (β8 + x)(β6 + x)(β12 + x)(β9 + x)g(x)
< previous page | page_323 | next page >


page_324

< previous page | page_324 | next page >

Page 324

6.2.3 (a) β2 (b) β5 (c) β4

6.2.7 (a) n = 3, k = 1, d = 3 and |C| = 4 (b) G = [ββ21]

message | codeword c | f(c)
0 | 000 | 000000
1 | β β2 1 | 011110
β | β2 1 β | 111001
β2 | 1 β β2 | 100111

6.2.8 (a) n = 7, k = 3, d = 5 and |C| = 83 = 512 (b) g(x) = β6 + β5 x + β5 x 2 + β2 x 3 + x 4

6.2.9 (a) β + β2 x + x 2 = (β3 + x)(β4 + x) = (1 + x)(β + x) (b) 1 + β6 x + x 2 = (β3+x)(β4+x) (c) β3+βx+x 2+β3 x = (d) β10+β3 x β6 x 2 β3 x 3+x 4 = (β+x)(β2+x)(β4+x) (e) β21+ β24 x+β16 x 2+β24 x 3+β9 x 4+β10 x 5+x 6 = (β+x)(β2+x)...(β6+x)

6.3.5 (a) 00ββ5β3β2β13β10β000000 (b) 1β4β2ββ12β910ββ5β3β2β13β10β (c) ββ10β70β12β3β310000000

6.3.6 (a)001β8β11β3β50000000 (b) 0β10β3β6β130β8β11β3β500000 (c) ββ10β70β12β3β310000000

6.3.8 (a) 0β20000000000000 (b) 00β00β3000000000 (c) 100000000000000 (d) β511100000000000 (e) β10β30001000010000 (f) β20000β20000β20000

6.5.4 (a) (βx) (b) (β2 x)(β5+x) (c) (β5+x)(β10+x)
(d) (1+x)(β+x)(β2+x)(β3+x) (e) (1+x)(β+x)(β5+x)(β10+x)
(f) (1+x)(β5+x)(β10+x)

In the following tables, for pi and qi the symbol * represents the zero field element and i represents βi.

(a)
< previous page | page_324 | next page >


page_325

< previous page | page_325 | next page >
Page 325
< previous page | page_325 | next page >


page_326

< previous page | page_326 | next page >

Page 326

6.6.9 (a) 1010 1111 1111 0011 1001 0000 0000 (b) 1001 1010 0000 0011 1010 0011 1001 (c) 0101 1001 0000 1100 1001 1100 0101 (d) 0000 1010 1111 1111 0011 1001 0000

6.6.10 Decode f(w) to f(c), where c is: (a) β10β12β7β3β12β8β8β20000000|(b) β100ββ7β70β0β20000000 (c) 0β12β14β4β2β8β200000000

6.6.11 Decode f(w) to f(c) where c = β7β71β9ββ10β810000000.

7 Burst error-correcting codes

7.1.5 C is not a 2 error-correcting code since it has only 32 cosets.

7.1.6 C is not 3 error-correcting code since it has only 64 cosets.
< previous page | page_326 | next page >


page_327

< previous page | page_327 | next page >

Page 327

7.1.13 (a) 101100000001000 (c) 100000101010011 (e) 00000111100100

7.1.14 (a) 010100000010010 (c) 001110000000100 (e) 000000011111010

7.2.4 (a) 1000110 0110110 1110000 0011100 0110110 0001111 (b) 10 01 01 00 11 11 00 10 10 11 01 01 00 00 00 10 10 01 11 11 01 (c) 101 011 011 000 110 110 000 000 010 110 101 111 011 001

7.2.8 (a) 1******00*****110****0110***00101**011011*
(b) 1******0******10*****01*****010****001****

7.2.9 The codewords are transmitted in order with no interleaving.

7.2.12 (a) 01 10 11 11 10 01 10 11 11 01 01 00 01 00 11 01 10 11 10 10 11 00 01 01 (b) 011 101 111 110 100 010 001 100 111 101 110 011

7.2.13 (a) m 1=0000, m 2=0011, m 3=0000 (b) m 1=1000, m 2=0110, m 3=0011.

8 Convolutional Codes

8.1.7 (a) 11101001... (b) 0010111...

8.1.12 (a) 000, 0010000 (b) 001, 1110000

8.1.14 (a) 000, 0010000 (b) 000, 100

8.2.2 (a) c(x) = (1+x + x 4+x 6, 1+x + x 2+x 4+x 5+x 6, 1+x 2+x 5+x 6)
(b) c(x) = (1+x 2+x 6, 1+x 3+x 5+x 6, 1+x + x 2+x 3+x 4+x 5+x 6)
(c) c(x) =

8.2.3 (a) c(x)=(1+x + x 2+x 3+x 6, 1+x 2+x 5+x 6) (b) c(x)=(1+x + x 5+ x 6+x 7, 1+x 7) (c) c(x)=(1+x 2+

8.2.6 The interleaved form of the codewords are as follows: (from 8.2.2) (a) 111 110 011 000 110 011 111... (b) 111 001 101 011 001 011 111 (c) 111 001 010 101 101 101 101... (from 8.2.3) (a) 11 10 11 10 00 01 11... (b) 11 10 00 00 00 10 10 11... (c) 11 01 10 11 01 11 01 11 01...

8.2.11 (a)
< previous page | page_327 | next page >


page_328

< previous page | page_328 | next page >

Page 328

(b)(i) 11 01 00 01 11 00 00... (ii) 11 10 01 10 11 00 00...
(c)(i) 1 0 1 0 0 0... (d)(ii) 0 1 1 1 1 0 1...

8.2.12 (a)

(b)(i) 11 10 11 00 10 11 11 00 00...
(ii) 11 01 01 11 01 11 11 00 00... (iii) 11 01 10 01 01 01...
(c)(i) 1 0 1 0 1 1 1... (ii) 0 1 1 1 1 0 0...

8.3.1 (a)m = 1010101...= [image] (b)1 * 1 * 1 *... (c) *000...

8.3.2 (a) gcd = 1+x; the loop on state 111 is a zero weight cycle (b) gcd = 1; not catastrophic (c) gcd = 1+x + x 2; (0110, 1011, 1101) is a zero weight cycle.

8.3.3 (a)5 (b)6 (c)7

8.3.6 (a) τ(a) = 2, τ(2) = 6 (b) τ(1) = 2, τ(2) = 6 (c) τ(1) = 2, τ(2) = 9, τ(3) = 13

8.4.4

State s | t = 8 | t = 9 | t = 10 | t =11 | t = 12
000 | 3,00000** | 3,000000* | 3,0000000 | 3,0000000 | 3,0000000
100 | 5,100**** | 3,1001001 | 5,100**** | 3,1001110 | 5,100****
010 | 4,0100100 | 4,0101001 | 4,0100100 | 4,0101110 | 4,0100111
110 | 4,1100100 | 4,1101001 | 4,1100100 | 4,1101110 | 4,1100111
001 | 3,0010011 | 5,001**** | 3,0011100 | 5,001***0 | 5,001*1*1
101 | 3,1010011 | 5,101**** | 3,1011100 | 5,101***0 | 5,101*1*1
011 | 4,011*0** | 2,0111001 | 4,0111*0* | 4,0111010 | 5,0111001
111 | 2,1110011 | 4,111*0** | 4,1110100 | 4,1110010 | 4,1110111
Decode to: | 1 | 1 | 0 | 0 | 0

8.4.5 (a)m = 0 0 0 (b)m = 1 0 0
< previous page | page_328 | next page >


page_329

< previous page | page_329 | next page >

Page 329

8.4.14 (b)

State | Output |  |
s | X 3 = 0 | X 3 = 1 | t = 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8
000 | 00 | 11 | ∞ | ∞ | ∞ | 7 | 6 | 6 | 6 | 6
100 | 11 | 00 | 2 | ∞ | ∞ | 5 | 4 | 5 | 5 | 6
010 | 10 | 01 | ∞ | 3 | ∞ | 4 | 6 | 5 | 6 | 6
110 | 01 | 10 | ∞ | 3 | ∞ | 4 | 6 | 5 | 6 | 6
001 | 11 | 00 | ∞ | ∞ | 5 | 4 | 5 | 5 | 6 | 6
101 | 00 | 11 | ∞ | ∞ | 3 | 6 | 4 | 6 | 5 | 6
011 | 01 | 10 | ∞ | ∞ | 4 | 5 | 5 | 6 | 6 | 7
111 | 10 | 01 | ∞ | ∞ | 4 | 5 | 5 | 6 | 6 | 7

d = 6, τ(1) = 2, τ(2) = 6.

9 Reed-Muller and Preparata Codes

9.1.3 (a) fI(x) = (x 0+1)(x 3+1) (b) vI = 1000100000000000, fI(x) = (x 0+ 1)(x 1+1)(x 3+1) (c) fI(x) = (x 1+1) (d) vI = 1111000000000000, fI(x) = (x2+1)(x3+1) (e) vI = 1, fI(x) = 1 (f) 100...0, fI(x) =

9.1.4 (a) fI(x) = (x 0+1)(x 4+1) (c) fI(x) = (x 1+1) (d) fI(x) = (x 1+1)(x 2+ 1)(x 4+1) (e) vI = 11...1, fI(x) = 1 (f) vI = 100...0, fI(x) =

9.1.5 There are |I| coordinates which must be 0 and there are two choices for each of the other m –; |I| coordinates in HI.

9.1.6 Since all vI have even weight except for vIm, v will have even weight if and only if

9.1.9 (a)

9.1.12 (a) c = v 2+v 0 = 0101 1010 0101 1010
(b) c = v 0,1 = 1000 1000 1000 1000 (c) c = v 2+v 0,3 = 0101 1010 1111 0000

9.2.7 (a) 0 1000 000001 (b) 0 0000 0 11000 (c) 1 1001 100000 (d) 1 1111 111111 (e) 0 0100 000100 (f) 0 0101 010000 (g) 0 0000 000010 (h) 0 0110 00000 (i) 1 000 1 000 101

9.2.8 (a) 0 00000 0000000100 (b) 0 00100 1000100001 (c) 1 00000 0000010000 (d) 1 00100 1100000000 (e) 0 01001 0000000100 (f) 0 10010 0000000000 (g) ask for retransmission
< previous page | page_329 | next page >


page_33

< previous page | page_33 | next page >

Page 33

These examples illustrate how to obtain a basis for the code C = 〈S〉 generated by a nonempty subset S of Kn. To find a basis for the dual code C⊥, extract a largest linearly independent subset from C⊥ following the procedure in Example 2.3.3

Exercises

2.3.7 For each set in Exercise 2.2.3 find a basis B for the code C = 〈S〉 and a basis B⊥ for the dual code C⊥.

The set B = {110, 011, 111} is not the only largest linearly independent subset of S = {110, 011, 101, 111} (see Example 2.3.6). The set B 1 = {110, 101, 111} is also such a subset of S. Thus B 1 is also a basis for the code C = 〈S〉.

In general a vector space usually has many bases. However, all bases for a vector space contain the same number of elements. The number of elements in any basis for a vector space is called the dimension of the space.

The dimension of Kn is n, since the set of all words of length n and weight one is a basis for Kn. At the other extreme, the basis of the subspace {0} is [image] and so has dimension 0.

Exercises

2.3.8 Find the dimensions of each code C = 〈S〉 and its dual C⊥ in Exercise 2.2.3 (see also Exercise 2.2.7).

A basis provides an efficient way to describe a linear code. For any vector space V, if {v 1, v 2,...,vk} is a basis for V, then every vector w in V can be expressed as a unique linear combination of the basis vectors v 1, v 2,...,vk; that is, there exist unique scalars a 1, a 2,...,ak such that w = a 1 v 1+a 2 v 2+...+ak vk.

Example 2.3.9 We write w = 011 as a unique linear combination of the words in the basis {110, 001, 100} for K 3. We seek digits a, b, c such that

a(110) + b(001) + c(100) = 011.

This yields the scalar equations

a+c = 0, a = 1, b = 1,

which have the unique solution a = b = c = 1. Thus 011 = 1(110) + 1(001) + 1(100).

Exercises

2.3.10 Write each of the following words in K 4 as a unique linear combination of the words in the basis {1000, 1100, 1110, 1111}:

(a) 0011

(b) 1010

(c) 0111

(d) 0001

(e) 0000.
< previous page | page_33 | next page >


page_330

< previous page | page_330 | next page >

Page 330

9.3.10 (a) (ii) 10011010 11110011 (iii) 01100101 11110011 (c) (ii) 11000110 10101111 (iii) 11001001 11100111

9.1.11 (a) If α = 0 then αU = {0}, so |αU| is odd, so [X(U), X(V)] does not satisfy (i) of Definition 9.3.3.

9.3.17 (a) 01000001 01110100 (b) 00001001 01001110 (c) 00000011 11010010

9.4.6 (a) 10101001 11011011 (b) 10101001 00100100 (c) 11111111 11111111 (d) 11111111 00000000 (e) 00000000 11111111

9.4.7 (a) 10100...0 00000100010...0 (b) 10100...0 00 ... 0

9.4.8 (a) 31 (b) 21

9.5.3 (a) 10000001 11101000 (b) 00011110 01000010 (c) 00000101 10100110 (d) 01000010 00011110 (e) 11101000 10000001 (f) 10011001 01111101 (g) Ask for retransmission (h) 10100101 10010000 (i) 11101101 01010101 (j) 10111011 01101010 (k) 01010101 11101101 (n) 01101010 10111011 (o) 10100101 10010000

9.5.4 (a) 11000 11000 10000 00000 00000 10000 11 00011 11000 00000 01000 00011 00100 00 (b) 10100 00000 00000 00000 00000 00000 00 00000 10001 00000 00000 01010 10111 00

9.5.5 No

10 Classical Cryptography

10.2.5 The 'VHV' fragments suggest a keylength dividing 16. Since plaintext 'an' maps to ciphertext 'AE', it follows that the key contains 'AR'. After showing that the keylength must be greater than 2, consider the case that the length is 4. The information concerning the 'AE' fragment gives key '?AR?', where the questionmark denotes an unknown character. Now use the information concerning the pair of common three-letter words to make an educated guess at the key.

10.2.8 There are scenarios where compression followed by encryption could aid cryptanalysis (for example, if compression places a known header on the out-put, then a known-plaintext attack may be possible, depending on the encryption method). However, the general recommendation is for compression first. Compression may be more effective on plaintext than ciphertext. If encryption or transmission is time-expensive, then compressing first may improve throughput. Attacks based on source redundancy may be frustrated by compression before encryption. See Boyd [16] and the references therein.

10.3.1 m = (m 0, m 1) = (1110, 0000) 10.3.3 m 1 = m 3 [image] f k2(m 2) = 1010 [image] f 1(1011) = 0000 and m = (m 0,m 1) = (1110, 0000).
< previous page | page_330 | next page >


page_331

< previous page | page_331 | next page >

Page 331

10.3.4 (a) In CBC, decryption is [image] Only mj and mj+1 depend on cj.

10.3.5 (a) Note that k 2 = [image] (m) ⊕ Ek(m). For each 0 ≤ i < 256, set j = DES i(m1) ⊕ c1. If DES i(m2) ⊕ j = c2, then it is likely that (i, j) = (k 1,k 2). At most 257 DES operations are required to find a candidate.

10.3.7 It is not known if the complementation property can be used to improve exhaustive key search under a known-plaintext attack. For a chosen-plaintext attack, obtain pairs (m,c1) and [image] and use the complementation property to eliminate two candidate keys with each DES operation.

11 Topics in Algebra and Number Theory

11.1.14 Algorithm 11.1.7 with n = 576 gives:

i | 0 | 1 | 2 | ... | 8
ki | 0 | 0 | 1 | ... | 1
A | 47 | 472 mod n = 481 | 4812 mod n = 385 | ... | 1932 mod n = 385
b | 1 | 1 | 1.385 mod n = 385 | ... | 385.385 mod n = 193

and 47332 mod 576 = 193.

11.1.16 The set {2, 6, 7, 8} of generators can be found by exhaustive calculation; e.g. and 25 [image], so 2 has order 10 since the order of an element must divide φ(11) = 10. If [image] has a generator α, then it can be shown that αi is a generator iff (i,φ(n)) = 1. It follows that if [image] is cyclic, then there are φ(φ(n)) generators. In this exercise, there are φ(φ(11)) = φ(10) = 4 generators, given by 2i for i [isin] {1, 3, 7, 9}.

11.1.21 (a) Use the division algorithm to write x = q·ord (a)+r, 0 ≤ r < ord(a). Suppose [image] (mod n). Since r < ord(a), it follows (from the definition of order) that r = 0 and hence ord(a) | x.

11.1.22 Use the fact that [image] has a generator and apply Exercise 11.1.20.

11.2.7 Calculate x 2 mod 30 for each x ∈ [image] ={1, 7, 11, 13, 17, 19, 23, 29} to obtain Q 30 = {1, 19}. (It suffices to consider x < 30/2 since n –; x ≡ –; x (mod n).)

11.2.8 (156/235) = –;1, (1833/587) = (72/587) = (2332/587) = (2/587)3 (3/587)2 = –;1.

11.2.13 Apply Euler's criterion.

11.3.5 If a is not an Euler witness, then [image] Consider the square of each quantity in the congruence.

A test for compositeness which searches for Fermat witnesses can be constructed; however, there are composite n (known as Carmichael numbers) for which there are no Fermat witnesses in [image].
< previous page | page_331 | next page >


page_332

< previous page | page_332 | next page >

Page 332

11.4.5 m = [image] and q(z) = (z+32)2 –; 1081. The factor base is B = {–;1,2,3,5,11} since (n/7) = –;1. The first few values of z for which q(z) factors over B are given in the following table.

z | a = z+m | b = q(z) | factorization
–;1 | 31 | –;120 | –;23·3·5
1 | 33 | 8 | 23
2 | 34 | 75 | 3·52
–;3 | 29 | –;240 | –;24·3·5

The relations for z ∈ {–;1,1,–;3} give x2 ≡ y2 (mod n) for x = 31·33·29 and y = 25·3·5; unfortunately, x ≡ 480 ≡ y (mod n). No other combination produces a perfect square, so more entries must be generated.

11.4.6 This problem appears in [63]. Values x and y satisfying x2 ≡ y2 are found corresponding to z ∈ {0, 1, –;2} and to z ∈ {–;1, 4, –;6}; the first of these gives x ≡ –; y.

11.4.8 Note that 179 ≡ 3 (mod 11) and 179 ≡ 9 (mod 17). The congruence x 2 ≡ 3 (mod 11) has solutions x = ±3(11+1)/4 since 11 ≡ 3 (mod 4). Alt hough we have not described an algorithm for solving x 2 ≡ 9 (mod 17), the congruence can easily be solved by inspection. Gauss' algorithm gives 71 as one of the roots of 179.

11.4.10 The value p+q can be calculated from n and φ. Consider the equation (x –; p)(x –; q) = 0.

11.5.3 The table of (j,αj) pairs is:

j | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9
αj mod p | 1 | 5 | 25 | 28 | 43 | 21 | 8 | 40 | 6 | 30

Calculate βα–;im mod p until a match is found:

Hence, βα–;im ;αj for i = 6 and j = 8, giving log54 = 68 in [image] 97.

11.5.6 (a) If λ = 2iλ′ with λ′ odd, then we may assume without loss of generality that 2i | φ(p). Consider an element a ∈ [image] with order φ(p) as an element of [image] and order φ(q)/2 as an element of [image].
< previous page | page_332 | next page >


page_333

< previous page | page_333 | next page >

Page 333

(b) Show that λ = ord(a)λ′ and x = 2i ord (a)x′ for some i ≥ 0 and odd λ′ and x′. Then [image] (mod n).

12 Public-key Cryptography

12.1.7 The adversary may calculate h(kxy) = f (M, y) for any single block y.

12.1.8 Note that 2pq | xi –; xj. If, for example, d = (x1–;x2, x1–;x3) < n, then either 2pq or 4pq has been recovered. Since n = (2p+1)(2q+1), it is then possible to efficiently recover p and q from n and d.

12.2.2 (a) The Euclidean algorithm gives d = 233. (b) Algorithm 11.1.7 can be used to show c = me mod pq = 921.

12.2.3 The question may be answered directly by checking the formula for the number of unconcealed messages. More generally, for any RSA modulus n = pq, it is easy to show that there is an exponent e with 1 < e < φ(n), (e,φ(n)) = 1, and me ≡ m (mod n) for all m. Consider e = 1+jφ(n)|(p –; 1, q –; 1) for some 1 ≤ j < (p –; 1, q –; 1). The exercise is the case j = (p –; 1, q –; 1)/2.

12.2.4 The system of congruences x ≡ ci (mod ni) has a solution x < n1 n2 n3. Since m < ni, it follows that x = m 3. The (integer) cube root of x can be recovered efficiently, giving m. (In the unlikely event that the moduli ni are not pairwise relatively prime, then a modulus can be factored.)

12.2.6 (a) If k < 1, then 1 = ed –; kφ(n) ≥ ed, a contradiction. Since d < φ(n), it follows that 1 = ed –; kφ(n) < (e –; k)φ(n) and k < e. (b) [image] [image].

12.2.7 (a) n –; φ(n) = n –; (n –; p –; q+1) < p+q < 3√n. (b) [image] [image] (c) Compute the close approximations [image] The candidate d′ can be checked by finding φ′ from ed′ –; k′φ′ = 1 and (if φ′ ∈ [image]) attempting to factor n using Exercise 11.4.10.

12.2.10 If ed ≡ 1 (mod λ), then If ed ≡ 1 (mod p –; 1) and med ≡ m (mod p). Similarly med ≡ m (mod q) and hence med ≡ m (mod n). Note that λ | φ(n), so using λ may result in a smaller d; however, if p and q are chosen at random, then (p –; 1, q –; 1) is expected to be small.

12.2.11 (a) The argument is essentially that in 12.2.10. (b) λ = 12, φ(n) = 48, φ(p,q) = 84, and λ | φ(p,q).

12.2.13 Select a message m > p and calculate c = me mod n as the chosen-ciphertext.

12.4.2 (a) x = 9 and (r,s) = (7, 13).

12.5.3 (b) If e = 1 is expected, then the choice X = S and y = 1 would satisfy the corresponding verification condition, although a transcript from such a session

< previous page | page_333 | next page >


page_334

< previous page | page_334 | next page >
Page 334 would not look genuine. Instead, consider the case that X = Sx is selected. 12.5.5 (a) Consider the scenario where the adversary intercepts αa and αb, replacing each by the value 1.
< previous page | page_334 | next page >


page_335

< previous page | page_335 | next page >

Page 335

Bibliography

[1] | Derek Atkins, Michael Graff, Arjen K. Lenstra, and Paul C. Leyland. The magic words are squeamish ossifrage. In Josef Pieprzyk and Reihanah Safavi-Naini, editors, Advances in Cryptology – ASIACRYPT '94, volume 917 of Lecture Notes in Computer Science, pages 263–277. Springer-Verlag, 1995.
[2] | Eric Bach. Discrete logarithms and factoring. Technical Report UCB/CSD 84/186, University of California Berkeley, Computer Science Division, June 1984.
[3] | Henry Beker and Fred Piper. Cipher Systems: The Protection of Communication. J. Wiley & Sons, New York, 1982.
[4] | Mihir Bellare and Phillip Rogaway. Optimal asymmetric encryption. In Alfredo De Santis, editor, Advances in Cryptology – EUROCRYPT '94, volume 950 of Lecture Notes in Computer Science, pages 92–111. Springer-Verlag, 1995. A revised version is available via http://www-cse.ucsd.edu/users/mihir/.
[5] | E. R. Berlekamp. Algebraic Coding Theory. McGraw-Hill, 1968.
[6] | R. E. Blahut. Theory and Practice of Error Control Codes. Addison-Wesley, 1983.
[7] | I. F. Blake and R. C. Mullin. An Introduction to Algebraic and Combinatorial Coding Theory. Academic Press, 1976.
[8] | Ian F. Blake, G. Seroussi, and Nigel P. Smart. Elliptic Curves in Cryptography, volume 265 of London Mathematical Society Lecture Note Series. Cambridge University Press, 1999.
[9] | Matt Blaze, Whitfield Diffie, Ronald L. Rivest, Bruce Schneier, Tsutomu Shimomura, Eric Thompson, and Michael Wiener. Minimal key lengths for symmetric ciphers to provide adequate commercial security: A report by an ad hoc group of cryptographers and computer scientists. Available through http://www.bsa.org/, January 1996.
[10] | Daniel Bleichenbacher. Generating ElGamal signatures without knowing the secret key. In Maurer [59], pages 10–18. A revised version correct-
< previous page | page_335 | next page >


page_336

< previous page | page_336 | next page >

Page 336

 | ing Corollary 2 is available from the Information Security and Cryptology Research Group, ETH-Zurich, ftp://ftp.inf.ethz.ch.
[11] | M. Blum. Coin flipping by telephone: a protocol for solving impossible problems. In Proceedings of the 24th IEEE Computer Conference (Comp-Con), pages 133–137, 1982.
[12] | Dan Boneh. Twenty years of attacks on the RSA cryptosystem. Notices of the AMS, 46(2):203–213, February 1999. Available via http//theory.stanford.edu/~dabo/.
[13] | Dan Boneh, Richard A. DeMillo, and Richard J. Lipton. On the importance of checking cryptographic protocols for faults. In Fumy [35], pages 37–51. The extended abstract is expanded in ''On the importance of eliminating errors in cryptographic computations," available via http://theory.stanford.edu/~dabo/.
[14] | Dan Boneh and Glenn Durfee. New results on the cryptanalysis of low exponent RSA. In J. Stern, editor, Advances in Cryptology – EUROCRYPT '99, volume 1592 of Lecture Notes in Computer Science, pages 1–11. Springer-Verlag, 1999. Available via http://theory.stanford.edu/~dabo/.
[15] | Dan Boneh and Ramarathnam Venkatesan. Breaking RSA may be easier than factoring. In K. Nyberg, editor, Advances in Cryptology – EUROCRYPT '98, volume 1403 of Lecture Notes in Computer Science, pages 59–71. Springer-Verlag, 1998. Available via http://theory.stanford.edu/~dabo/.
[16] | Colin Boyd. Enhancing security by data compression: theoretical and practical aspects. In D. W. Davies, editor, Advances in Cryptology – EUROCRYPT '91, pages 267–280. Springer-Verlag, 1991.
[17] | Gilles Brassard, editor. Advances in Cryptology – CRYPTO '89, volume 435 of Lecture Notes in Computer Science. Springer-Verlag, 1989.
[18] | Gilles Brassard and Claude Crépeau. Sorting out zero-knowledge. In J.J. Quisquater and J. Vandewalle, editors, Advances in Cryptology – EUROCRYPT '89, volume 434 of Lecture Notes in Computer Science, pages 181–191. Springer-Verlag, 1990.
[19] | David Chaum, Jan-Hendrik Evertse, and Jeroen van de Graaf. An improved protocol for demonstrating possession of discrete logarithms and some generalizations. In David Chaum and Wyn L. Price, editors, Advances in Cryptology – EUROCRYPT '87, volume 304 of Lecture Notes in Computer Science, pages 127–141. Springer-Verlag, 1988.
[20] | David Chaum, Jan-Hendrik Evertse, Jeroen van de Graaf, and René Peralta. Demonstrating possession of a discrete logarithm without revealing it. In Andrew M. Odlyzko, editor, Advances in Cryptology – CRYPTO
< previous page | page_336 | next page >


page_337

< previous page | page_337 | next page >

Page 337

 | '86, volume 263 of Lecture Notes in Computer Science, pages 200–212. Springer-Verlag, 1987.
[21] | Clifford C. Cocks. A note on 'non-secret encryption'. Technical report, Communications Electronics Security Group (CESG), November 1973. Available via http://www.cesg.gov.uk.
[22] | Don Coppersmith. Cheating at mental poker. In Williams [98], pages 104–107.
[23] | Don Coppersmith. The Data Encryption Standard (DES) and its strength against attacks. IBM Journal of Research and Development, 38(3):243–250, May 1994.
[24] | Don Coppersmith, Matthew Franklin, Jacques Patarin, and Michael Reiter. Low-exponent RSA with related messages. In Maurer [59], pages 1–9.
[25] | Richard A. DeMillo, Georgie I. Davida, David P. Dobkin, Michael A. Harrison, and Richard J. Lipton. Applied Cryptology, Cryptographic Protocols, and Computer Security Models. Proceedings of Symposia in Applied Mathematics. American Mathematical Society, Providence, 1983. Lecture notes for the AMS short course Cryptology in Revolution: Mathematics and Models, San Francisco, 1981.
[26] | Whitfield Diffie. The first ten years of public key cryptography. In Simmons [81], chapter 3, pages 135–175.
[27] | Whitfield Diffie and Martin E. Hellman. New directions in cryptography. IEEE Transactions on Information Theory, 22(6):644–654, 1976.
[28] | Whitfield Diffie and Martin E. Hellman. Exhaustive cryptanalysis of the NBS Data Encryption Standard. Computer, 10(6):74–84, June 1977.
[29] | Taher ElGamal. A public key cryptosystem and a signature scheme based on discrete logarithms. IEEE Transactions on Information Theory, 31(4):469–472, July 1985.
[30] | J. H. Ellis. The possibility of secure non-secret digital encryption. Technical report, Communications Electronics Security Group (CESG), January 1970. Available via http://www.cesg.gov.uk.
[31] | J. H. Ellis. The history of non-secret encryption. Technical report, Communications Electronics Security Group (CESG), December 1997. Available via http://www.cesg.gov.uk.
[32] | David C. Feldmeier and Philip R. Karn. Unix password security—ten years later. In Brassard [17], pages 44–63.
[33] | Steven Fortune and Michael Merritt. Poker protocols. In G. R. Blakley and David Chaum, editors, Advances in Cryptology – CRYPTO '84, volume 196
< previous page | page_337 | next page >


page_338

< previous page | page_338 | next page >

Page 338

 | of Lecture Notes in Computer Science, pages 454–464. Springer-Verlag, 1985.
[34] | Electronic Frontier Foundation. Cracking DES: Secrets of Encryption Research, Wiretap Politics, and Chip Design. Distributed by O'Reilly and Associates, 1998.
[35] | Walter Fumy, editor. Advances in Cryptology – EUROCRYPT '97, volume 1233 of Lecture Notes in Computer Science. Springer-Verlag, 1997.
[36] | R. G. Gallager. Information Theory and Reliable Communication. John Wiley and Sons, 1968.
[37] | Simson Garfinkel. PGP: Pretty Good Privacy. O'Reilly & Associates, 1995.
[38] | W. J. Gilbert. Modern Algebra with Applications. Wiley, 1976.
[39] | Ian Goldberg and David Wagner. Randomness and the Netscape browser. Dr. Dobb's Journal, pages 66–70, January 1996.
[40] | Shafi Goldwasser, Silvio Micali, and Charles Rackoff. The knowledge complexity of interactive proof systems. SIAM Journal on Computing, 18(1):186–208, February 1989.
[41] | Martin Handford. Where's Waldo? Little, Brown, Boston, 1987.
[42] | G. H. Hardy and E. M. Wright. An Introduction to the Theory of Numbers. Oxford Clarendon Press, second edition, 1945.
[43] | R. Hill. A First Course in Coding Theory. Oxford University Press, 1986.
[44] | Don Johnson and Alfred Menezes. The Elliptic Curve Digital Signature Algorithm (ECDSA). CORR 99-34, University of Waterloo, Canada, August 1999. Available from http://www.cacr.math.uwaterloo.ca.
[45] | D. S. Jones. Elementary Information Theory. Oxford University Press, 1979.
[46] | Antoine Joux and Reynald Lercier. State-of-the-art in implementing algorithms for the (ordinary) discrete logarithm problem. The 3rd workshop on Elliptic Curve Cryptography (ECC '99), University of Waterloo, http://www.cacr.math.uwaterloo.ca, November 1–3 1999.
[47] | Marc Joye, Arjen K. Lenstra, and Jean-Jacques Quisquater. Chinese remaindering based cryptosystems in the presence of faults. Journal of Cryptology, 12(4):241–245, Autumn 1999.
[48] | David Kahn. The Codebreakers: The Story of Secret Writing. Scribner, New York, revised edition, 1996.
[49] | Joe Kilian and Phillip Rogaway. How to protect DES against exhaustive key search. In Koblitz [51], pages 252–267. Available via http://www.cs.ucdavis.edu/~rogaway/; a summary appears in [73].
< previous page | page_338 | next page >


page_339

< previous page | page_339 | next page >

Page 339

[50] | Neal Koblitz. A Course in Number Theory and Cryptography. Springer, second edition, 1994.
[51] | Neal Koblitz, editor. Advances in Cryptology – CRYPTO '96, volume 1109 of Lecture Notes in Computer Science. Springer-Verlag, 1996.
[52] | Paul Kocher, Joshua Jaffe, and Benjamin Jun. Differential power analysis. In Michael Wiener, editor, Advances in Cryptology – CRYPTO '99, volume 1666 of Lecture Notes in Computer Science, pages 388–397. Springer-Verlag, 1999.
[53] | Paul C. Kocher. Timing attacks on implementations of Diffie-Hellman, RSA, DSS, and other systems. In Koblitz [51], pages 105–113.
[54] | B. A. LaMacchia and A. M. Odlyzko. Computation of discrete logarithms in prime fields. Designs, Codes and Cryptography, 1(1):47–62, May 1991.
[55] | Arjen K. Lenstra and Eric R. Verheul. Selecting cryptographic key sizes. The 3rd workshop on Elliptic Curve Cryptography (ECC '99), University of Waterloo, http://www.cacr.math.uwaterloo.ca, November 1–3 1999.
[56] | R. Lidl and H. Neiderreiter. Finite Fields. Cambridge University Press, 1984.
[57] | S. Lin and D. J. Costello, Jr. Error Control Coding: Fundamentals and Applications. Prentice-Hall, 1983.
[58] | F. J. MacWilliams and J. J. A. Sloane. The Theory of Error-Correcting Codes. North-Holland, 1977.
[59] | Ueli Maurer, editor. Advances in Cryptology – EUROCRYPT '96, volume 1070 of Lecture Notes in Computer Science. Springer-Verlag, 1996.
[60] | R. J. McEliece. The Theory of Information and Coding. Addison-Wesley, 1977.
[61] | R. J. McEliece. Finite Fields for Computer Scientists and Engineers. Kluwer Academic Publishers, 1987.
[62] | Alfred J. Menezes. Elliptic Curve Public Key Cryptosystems, volume 234 of Kluwer international series in engineering and computer science. Kluwer Academic Publishers, 1993.
[63] | Alfred J. Menezes, Paul C. van Oorschot, and Scott A. Vanstone. Handbook of Applied Cryptography. CRC Press, Boca Raton, 1996. Errata and a complete on-line copy of the book are available on http://www.cacr.math.uwaterloo.ca/hac/.
[64] | J. H. Moore. Protocol failures in cryptosystems. In Simmons [81], chapter 11, pages 541–558.
< previous page | page_339 | next page >


page_34

< previous page | page_34 | next page >

Page 34

Another important fact about vector spaces is that any linearly independent subset of a vector space is contained in a basis for the space. The next example shows how this works.

Example 2.3.11 The set S = {110, 001} is a linearly independent subset of K 3. We extend S to a basis for K 3. First we adjoin to S any known basis: {100, 010, 001} is a convenient basis to adjoint for K 3. The resulting list of words

110, 001, 100, 010, 001

is then reduced to a basis for K 3 according to the procedure in Example 2.3.3, solving for 100, 010 or 001.

Exercises

2.3.12 (a) Find a basis for K 4 which contains {1001, 1111}. (b) Extend {101010, 010101} to a basis for K 6.

We now come to two important theorems concerning dimension of linear codes. If a linear code C has dimension k and if {v1, v2, ..., vk} is a basis for C, then a word w in C can be written as

w = a1 v1 + a2 v2 +...+a k vk

for a unique choice of digits a1, a2, ..., ak. Since each a 1 is either 0 or 1, there are 2k choices for a1, a2, ..., ak, and hence 2k words in C.

Theorem 2.3.13 A linear code of dimension k contains precisely 2k codewords.

The next theorem can be proved using elementary results from the theory of systems of linear equations.

Theorem 2.3.14 Let C = 〈S〉 be the linear code generated by a subset S of Kn. Then (dimension of C) + (dimension of C⊥) = n.

Exercises

2.3.15 Check your answers in Exercise 2.3.8 with the equation in Theorem 2.3.14.

2.3.16 Let S be a subset of K 7, let C = 〈S〉 and assume C⊥ has dimension 3.

(a) Find the dimension of C = 〈S〉.

(b) Find the number of words in C.

2.3.17 Let S be a subset of K 8 and assume that {11110000, 00001111, 10000001} is a basis for C⊥. Find the number of words in C = 〈S〉.
< previous page | page_34 | next page >


page_340

< previous page | page_340 | next page >

Page 340

[65] | Moni Naor, Yael Naor, and Omer Reingold. Applied kid cryptography, or how to convince your children that you are not cheating. CRYPTO '98 rump session, August 1998.
[66] | W. W. Peterson and E. J. Weldon, Jr. Error-Correcting Codes. MIT Press, 1972.
[67] | V. Pless. Introduction to the Theory of Error-Correcting Codes. Wiley, 1982.
[68] | Jean-Jacques Quisquater, Louis Guillou, and Tom Berson. How to explain zero-knowledge protocols to your children. In Brassard [17], pages 628–631.
[69] | M. O. Rabin. Digitalized signatures and public-key functions as intractable as factorization. Technical Report 212, MIT Laboratory for Computer Science, 1979.
[70] | Rick Ramsey. All About Administering NIS+. SunSoft, second edition, 1994.
[71] | Ronald L. Rivest. Cryptography. In van Leeuwen [91], pages 719–755.
[72] | Ronald L. Rivest and Adi Shamir. How to expose an eavesdropper. Communications of the ACM, 27(4):393–395, April 1984.
[73] | Phillip Rogaway. The security of DESX. CryptoBytes, 2(2):8–11, Summer 1996. RSA Laboratories newsletter, http://www.rsa.com. The article is a summary of [49].
[74] | Kenneth H. Rosen. Elementary Number Theory and its Applications. Addison-Wesley, third edition, 1993.
[75] | Arto Salomaa. Public-Key Cryptography. Texts in theoretical computer science. Springer-Verlag, second edition, 1996.
[76] | Bruce Schneier. Applied Cryptography: protocols, algorithms, and source code in C. John Wiley & Sons, Inc., second edition, 1996.
[77] | Jennifer Seberry and Josef Pieprzyk. Cryptography: an introduction to computer security. Prentice Hall, 1989.
[78] | Adi Shamir. RSA for paranoids. CryptoBytes, 1(3):1–4, Autumn 1995. RSA Laboratories newsletter, http://www.rsa.com.
[79] | Adi Shamir, Ronald L. Rivest, and Leonard M. Adleman. Mental poker. In David A. Klarner, editor, The Mathematical Gardner, pages 37–43. Prindle, Weber, and Schmidt, Boston, 1981.
[80] | C. E. Shannon. A mathematical theory of communication. Bell System Technical Journal, 27:379–423 and 623–56, 1948.
< previous page | page_340 | next page >


page_341

< previous page | page_341 | next page >

Page 341

[81] | G.J. Simmons, editor. Contemporary Cryptology: the science of information integrity. IEEE Press, 1992.
[82] | Gustavus J. Simmons. The prisoners' problem and the subliminal channel. In David Chaum, editor, Advances in Cryptology – CRYPTO '83, pages 51–67, New York, 1984. Plenum Press.
[83] | Gustavus J. Simmons. The subliminal channel and digital signatures. In Thomas Beth, Norbert Cot, and Ingemar Ingemarsson, editors, Advances in Cryptology – EUROCRYPT '84, volume 209 of Lecture Notes in Computer Science, pages 364–378. Springer-Verlag, 1985.
[84] | Gustavus J. Simmons. Subliminal channels; past and present. European Transactions on Telecommunications, 5(4):459–473, July-August 1994.
[85] | Gustavus J. Simmons. Subliminal communication is easy using the DSA. In Tor Helleseth, editor, Advances in Cryptology – EUROCRYPT '93, volume 765 of Lecture Notes in Computer Science, pages 218–232. Springer-Verlag, 1994.
[86] | Douglas R. Stinson. Cryptography: Theory and Practice. CRC Press, Boca Raton, Florida, 1995.
[87] | Robert Sugarman. On foiling computer crime. IEEE Spectrum, 16(7):31–32, July 1979. This is the first of a series of articles: Martin E. Hellman, DES will be totally insecure within ten years, 32–39; Security Agency denies tampering with DES, National Security Agency, 39; George I. Davida, Hellman's scheme breaks DES in its basic form, National Science Foundation, 39; Walter Tuchman, Hellman presents no shortcut solutions to the DES, 40–41; Dennis Branstad, Hellman's data does not support his conclusion, National Bureau of Standards, 41.
[88] | Bradley Taylor and David Goldberg. Secure networking in the Sun environment. Technical Report 905, Sun Microsystems, January 1991.
[89] | A. Tietäväinen. On the nonexistence of perfect codes over finite fields. SIAM Journal on Applied Mathematics, 24:88–96, 1973.
[90] | Malcolm Turnbull. The Spycatcher Trial: the scandal behind the #1 best seller. Salem House Publishers, 1989. See [101].
[91] | J. van Leeuwen, editor. Handbook of Theoretical Computer Science. Elsevier Science Publishers, 1990.
[92] | J. H. van Lint. Introduction to Coding Theory. Springer-Verlag, 1982.
[93] | Michael Wiener. Efficient DES key search. In W. Stallings, editor, Practical Cryptography for Data Internetworks, pages 31–79. IEEE Computer Society Press, 1996. Reprinted from Crypto 93 rump session.
< previous page | page_341 | next page >


page_342

< previous page | page_342 | next page >

Page 342

[94] | Michael Wiener. Efficient DES key search—an update. In Cracking DES [34], chapter 11, pages 1–4.
[95] | Michael J. Wiener. Cryptanalysis of short RSA secret exponents. IEEE Transactions on Information Theory, 36(3):553–558, May 1990.
[96] | H. C. Williams. A modification of the RSA public-key encryption procedure. IEEE Transactions on Information Theory, 26(6):726–729, November 1980.
[97] | H. C. Williams. An M 3 public-key encryption scheme. In Williams [98], pages 358–368.
[98] | Hugh C. Williams, editor. Advances in Cryptology – CRYPTO '85, volume 218 of Lecture Notes in Computer Science. Springer-Verlag, 1985.
[99] | Malcolm J. Williamson. Non-secret encryption using a finite field. Technical report, Communications Electronics Security Group (CESG), January 1974. Available via http://www.cesg.gov.uk.
[100] | Malcolm J. Williamson. Thoughts on cheaper non-secret encryption. Technical report, Communications Electronics Security Group (CESG), August 1976. Available via http://www.cesg.gov.uk.
[101] | Peter Wright. Spycatcher: the candid autobiography of a senior intelligence officer. Viking, New York, 1987. See also [90].
[102] | Adam Young and Moti Yung. The dark side of ''black-box" cryptography, or: should we trust Capstone. In Koblitz [51], pages 89–103.
[103] | Adam Young and Moti Yung. Kleptography: using cryptography against cryptography. In Fumy [35], pages 62–74.
[104] | Philip R. Zimmermann. The Official PGP User's Guide. MIT Press, Cambridge, Massachusetts, 1995.
[105] | V. Zinoviev and V. Leontiev. The nonexistence of perfect codes over Galois fields. Problems of Control and Information Theory, 2(2):16–24, 1973.
< previous page | page_342 | next page >


page_343

< previous page | page_343 | next page >

Page 343

Index

Symbols

C⊥ (dual code), 30

C 23 (Golay code), 82

C 24 (extended Golay code), 77

G(r,m), 84

Gr,m 207

Jn (elements with Jacobi symbol 1), 263

O(·) (''big oh"), 254

P(r), 214

Qn (quadatic residues modulo n), 260

Qn (quadatic nonresidues modulo n), 260

n (pseudosquares), 263

GF(2r), 111

GF(2r)[x], 127

RM(r,m) (Reed-Muller code), 84, 207

RS(2r,δ) (Reed-Solomon code), 131

n (integers modulo n), 256

(invertibel elements of [image] n), 257

X(U), 213

≡ (congrunet), 256

(·,·) (greatest common divisor), 254

(÷) (Jacobi symbol), 261

lcm(·,·) (least common multiple), 259

· [image] (greatest integer), 24

⊕ (XOR, bitwise addition modulo 2), 236

ord(·) (order), 257

ø(·) (Euler phi), 256

π(v), 96

σA(x), 136

〈S〉, 29

τ(e), 191

mod, 94, 113, 256

θp(C), 63

{0, 1}* (bitstrings of finite length), 228

{0, 1}n (bitstrings of length n), 228

a | b (a divides b), 254

A

active adversary, 228

Adleman, Leonard M., 284, 302

Advanced Encryption Standard (AES), 245, 250

AES, see Advanced Encryption Standard algorithm, 253

baby-step giant-step, 274

Berlekamp-Massey decoding, 148

Berlekamp-Massey decoding with erasures, 155

decoding 2 error-correcting BCH codes, 124

decoding erasures in RS(2r, δ), 155

decoding P(r), 224

decoding RS(2r, δ), 137

decoding the extended Golay code, 80

decoding the RM(1, m) code, 89

efficient, 254

Euclidean, 255, 307

exponentiation by square-and-multiply, 258

finding a basis for C⊥, 39

Gauss' ~ for a system of congruences, 257

index calculus, 275

majority logic decoding of RM(r,m), 210

Miller-Rabin primality test, 266

Pollard's rho, 267

polynomial-time, 254

quadratic sieve, 270

random squares, 269
< previous page | page_343 | next page >


page_344

< previous page | page_344 | next page >

Page 344

running time, 253

Solovay-Strassen primality test, 266

square root, 272

truncated Viterbi decoding, 194

alphabet, 228

American National Standards Institute (ANSI), 250

ANSI, see American National Standards Institute

Arlington Hall, 250

Atkins, Derek, 271

attack, 230

active, 228, 298

chosen-ciphertext, 230, 285, 289

chosen-plaintext, 230–232, 240

ciphertext-only, 230, 231

exhaustive search of keyspace, 231, 243, 245, 248, 251

fault analysis, 287

frequency analysis, 231, 232

give-me-the-key, 230

intruder-in-the-middle, 298, 303

Kasiski examinination, 233

known-plaintext, 230, 244

meet-in-the-middle, 245, 249

passive, 228

tempest, 230

traffic analysis, 230

authentication, 227, 246–248

B

baby-step giant-step algorithm for discrete logarithms, 274

Bach, Eric, 277

basis, 32

BCH codes, 120

Beker, Henry, 250

Berlekamp-Massey, 148

Berson, Tom, 306

birthday paradox, 268

bit operation, 253

Blake, Ian F., 305

Blaze, Matt, 245

Bleichenbacher, Daniel, 296, 297

block cipher, 235

modes of operation, 246

blocklength, 235

Blum integer, 272, 291

Blum, M., 301

Boneh, Dan, 286, 287

Boyd, Colin, 330

Brassard, Gilles, 306

burst error-correcting code, 160

burst length, 159

C

Caesar, Julius, 231

canonical form, 207

Carmichael number, 331

catastrophic code, 189

CBC, see cipher-block chaining

Certification Authority (CA), 298

Chaum, David, 302

Chinese remainder theorem, 257

chosen-ciphertext attack, 230, 285, 289

chosen-plaintext attack, 230–232, 240

cipher, 228

block ~, 235

breakable, 229

Feistel, 238

memoryless, 235

monoalphabetic, 232

one-time pad, 236

polyalphabetic, 232

shift ~, 231

state ~, 235

stream ~, 235

substitution ~, 230, 237

Vernam, 236

Vigenère, 232–235, 237

see also public-key cryptography

cipher-block chaining (CBC), 246, 247, 250

CBC-MAC, 247, 249, 282

errors in ciphertext, 248

ciphertext, 228

ciphertext-only attack, 230, 231

CMLD, see complete maximum likelihood decoding

Cocks, Clifford C., 305

coin-tossing by telephone, 301

Cold War Experience, 236
< previous page | page_344 | next page >


page_345

< previous page | page_345 | next page >

Page 345

collision, see hash function

Communications-Electronics Security Group (CESG), 305

complete maximum likelihood decoding (CMLD), 13

composite integer, 254

compression and encryption, 238

computational security, 229

confusion, 239

congruence, 256

convolutional code, 179

Coppersmith, Don, 250, 286, 306

coset of C determined by u, 54

Crépeau, Claude, 306

crypt password algorithm, 247

cryptanalysis, 227

cryptology, 227

cyclic burst length, 159

cyclic code, 96

cyclic Hamming code, 118

cyclic shift, 96

D

Data Encryption Algorithm, see Data Encryption Standard

Data Encryption Standard (DES), 243–249

Challenge, 244, 250

cipher-block chaining, 250

complementation property, 249

controversy, 243, 250

Cracking DES, 244, 250

DESX extension, 249

exhaustive search of keyspace, 243–245, 248, 251

FIPS 46-3, 250

in crypt password algorithm, 247

meet-in-the-middle attack, 245, 249

S-box, 243, 250

triple-DES, 245, 249, 250

data integrity, 227

data origin authentication, see message authentication

decryption function, 228

DeMillo, Richard A., 287

DES, see Data Encryption Standard

DESCHALL, 244

designed distance, 131

DHP, see Diffie-Hellman problem

Diffie, Whitfield, 228, 243, 245, 249

Diffie-Hellman key agreement, 279, 298–299

active attack, 280, 298, 303

Diffie-Hellman problem (DHP), 279

relation to DLP and EL GAMAL, 279, 294

diffusion, 239

Digital Signature Algorithm (DSA), 295, 296

elliptic curve analogue, 305

subliminal channel, 304

see also Digital Signature Standard

Digital Signature Standard (DSS), 293, 295

dimension, 33

dimension k of the code C, 42

directed walk, 184

discrete logarithm, 274–277

baby-step giant-step, 274

DLP problem definition, 253, 274, 279

Gaussian integer method, 276

index calculus, 275

least significant bit, 277

number field sieve, 276

relation to DHP and EL GAMAL, 279, 294

relation to factoring, 277

distance invariant, 216

distributed.net, 244

Division algorithm, 92

divisor, 254

DLP, see discrete logarithm

DSA, see Digital Signature Algorithm

DSS, see Digital Signature Standard

dual code, 30

E

e-ready, 201

ECB, see electronic codebook mode

electronic codebook mode (ECB), 246

errors in ciphertext, 248
< previous page | page_345 | next page >


page_346

< previous page | page_346 | next page >

Page 346

Electronic Frontier Foundation (EFF), 244, 250

ElGamal public-key scheme, 293–297

explicit randomization, 294

message expansion, 294

problem definition, 294

relation to DLP and DHP, 294, 295

SETUP contamination, 305

signature, 294–296, 304

subliminal channel, 304

weak generator, 296

Ellis, J. H., 305

encryption function, 228

encryption scheme, 228

see also cipher, public-key cryptography

equivalence class, 256

equivalent, 51

erasure, 153

location number, 153

locator polynomial, 154

error

evaluator polynomial, 149

location number, 135

locations, 135

locator polynomial, 123, 136

magnitude, 135

Euclidean algorithm, 255, 307

extended, 255, 256

Euler phi function, 256

Euler witness, 264

Euler's criterion, 261

Evertse, Jan-Hendrik, 302

exhaustive decoding algorithm, 188

existential forgery, 295

extended code, 75

extended Golay code, 77

extended Preparata code, 214

F

factor base, 270, 275

factoring, 267–273

challenge, 269, 271

FACTOR problem definition, 253

Fermat, 273

number field sieve, 269

Pollard's rho, 267

quadratic sieve, 270

random squares, 269

relation to discrete logarithm, 277

relation to square root, 272

Fast Hadamard Transform, 88

Federal Information Processing Standard (FIPS), 250

feedback shift registers, 176

Feistel cipher, 238

Feldmeier, David C., 248

Fermat factorization, 273

Fermat witness, 266

Fermat's (little) theorem, 257, 260

Fiat-Shamir identification protocol, 300, 302

FIPS, see Federal Information Processing Standard

forgery, 247, 281–283, 285, 295, 296

Fortune, Steven, 306

Franklin, Matthew, 286

function space, 142

G

Galois field, 111

Garfinkel, Simson, 249, 290, 306

Gauss' algorithm, 257

generator, 97, 174, 257

generator matrix, 42

generator polynomial, 98

Gilbert-Varshamov bound, 69

Golay code, 82

Goldberg, David, 276, 306

Goldberg, Ian, 237

Goldwasser, Shafi, 300, 306

Grabeel, Gene, 250

Graff, Michael, 271

greatest common divisor, 254, 307

guard space, 201

Guillou, Louis, 306

H

Hamming bound, 66

Hamming code, 72

hash, 281

hash function, 281
< previous page | page_346 | next page >


page_347

< previous page | page_347 | next page >

Page 347

based on block cipher, 282

collision, 281, 283

collision resistance, 281, 283

Davies-Meyer, 282

keyed, 282

Matyas-Meyer-Oseas, 282

MD5, 283

preimage resistance, 281, 283

random search for collision, 283

SHA-1, 283, 295

unkeyed, 282

see also MAC, MDC

Hellman, Martin E., 228, 243

I

idempotent, 106

identification, 227, 247

IMLD, see incomplete maximum likelihood decoding

improper cyclic codes, 105

incomplete maximum likelihood decoding (IMLD), 13

index calculus method for discrete logarithms, 275–276

information digits, 50

interleaving

s-frame delayed, 166

cross, 167

to depth s, 163

International Organization for Standards (ISO), 250

irreducible, 105, 111

ISO, see International Organization for Standards

J

Jacobi symbol, 262

Jaffe, Joshua, 230

Johnson, Don, 305

Joux, Antoine, 276

Joye, Marc, 287

Jun, Benjamin, 230

K

Kahn, David, 229, 233, 235, 249

Karn, Philip R., 248

Kasiski examination, 233

Kerckhoffs' principle, 230

key agreement, 279

see also Diffie-Hellman key agreement

key distribution, 280

key management, 280

key-pair, 228

key-schedule, 239

keystream, 236

Kilian, Joe, 249

known-plaintext attack, 230, 244

Koblitz, Neal, 249, 305

Kocher, Paul C., 230

L

LaMacchia, B. A., 276, 306

least common multiple, 259

Legendre symbol, 261

length, 191

Lenstra, Arjen K., 271, 285, 287

Lercier, Reynald, 276

Leyland, Paul C., 271

linear code, 27

linear combination, 29

linear span, 29

linearly dependent, 31

linearly independent, 31

Lipton, Richard J., 287, 302

M

MAC, see message authentication code

majority logic decoding, 208, 210

Mars Global Surveyor, 183

Mars Pathfinder, 183

maximum distance separable (MDS), 67, 143

maximum likelihood decoding (MLD), 13

MDC, see modification detection code

MDS, see maximum distance separable

Menezes, Alfred, 249, 305, 306

mental poker, 302

Merritt, Michael, 306

message authentication, 227, 246
< previous page | page_347 | next page >


page_348

< previous page | page_348 | next page >

Page 348

message authentication code (MAC), 247, 282

CBC-MAC, 247, 249, 282

from an MDC, 283

message digest, 281

message polynomial, 102

message space, 228

Micali, Silvio, 300, 306

Miller-Rabin primality test, 266

minimal polynomial, 116

MLD, see maximum likelihood decoding

modification detection code (MDC), 282

modulo, 94, 113, 256

Moore, J. H., 297

multiple encryption, 245–246

N

Naor, Moni, 306

Naor, Yael, 306

National Bureau of Standards (NBS), 243

National Institute of Standards and Technology (NIST), 243, 250

National Security Agency (NSA), 243

VENONA project, 236, 250

NBS, see National Bureau of Standards

NDS, see New Data Seal

New Data Seal (NDS), 240–242, 250

chosen-plaintext attack, 240–242

NIST, see National Institute of Standards and Technology

non-repudiation, 227, 281

NSA, see National Security Agency

number-theoretic problem, 253

DHP, 279

DLP, 253, 274, 279

EL GAMAL, 294

FACTOR, 253

QRP, 263

RABIN, 291

RSA, 284

SQ ROOT, 272

O

Odlyzko, A. M., 276, 306

one-time pad, 236

one-way function, 280

oracle, 272

order, 115, 257

orthogonal, 30

orthogonal complement, 30

P

parity-check digit, 50

parity-check matrix, 45

passive adversary, 228

Patarin, Jacques, 286

Peralta, René, 302

PGP, see Pretty Good Privacy

Pieprzyk, Josef, 306

Piper, Fred, 250

Pollard's rho factoring method, 267–269

polynomial-time, 254

Porta, Giovanni Battista, 229

preimage resistance, see hash function

Pretty Good Privacy (PGP), 290, 306

primality test, 264–267, 288

Fermat, 331

Miller-Rabin, 266

Solovay-Strassen, 266

prime number, 254

prime number theorem, 254

primitive, 112, 114

primitive BCH codes, 131

prisoners' problem, 304

private key, 279

proper cyclic code, 105

protocol, 297–305

challenge-response, 300

coin-tossing by telephone, 301

Diffie-Hellman key agreement, 279

failure, 286, 297

Fiat-Shamir identification, 300

mental poker, 302

Shamir's 3-pass, 298

subliminal channel, 304

zero-knowledge, see zero-knowledge proof

provable security, 229, 290–293

pseudosquare, 263

public key, 279

public-key cryptography, 279
< previous page | page_348 | next page >


page_349

< previous page | page_349 | next page >

Page 349

compared with symmetric-key, 280

ElGamal, 293–297

Rabin, 290–292

RSA, 284–290

Williams, 292

see also protocol, signature

Q

QRP, see quadratic residuosity problem

quadratic nonresidue, 260

finding, 272

quadratic residue, 260

quadratic residuosity problem (QRP), 263, 301

quadratic sieve factoring method, 270

Quisquater, Jean-Jacques, 287, 306

R

Rabin public-key scheme, 290–292

problem definition, 291

relation to factoring, 291

Rabin, M. O., 290

Rackoff, Charles, 300, 306

Ramsey, Rick, 306

random number generator in Netscape, 237

seed, 237

random squares factoring method, 269–271

rank, 42

reduce (for computational problems), 272

reducible, 111

Reed-Muller code, 84, 207

Reed-Solomon code, 131

REF, see row echelon form

Reingold, Omer, 306

Reiter, Michael, 286

reliability, 63

residue class, 256

rho-diagram, 268

Rivest, Ronald L., 227, 245, 249, 284, 302, 303

Rogaway, Phillip, 249

roots of unity, 143

rot 13 encryption function, 231

row echelon form (REF), 36

RSA, 284–290

chosen-ciphertext attack, 285, 289

cycling attack, 288

decryption exponent, 284, 286, 287

encryption exponent, 284, 286

factoring challenge, 269, 271

fault analysis attack, 287

multiplicative property, 285

partial key-exposure, 287

problem definition, 284

related-message attack, 286

relation to factoring, 284, 288, 290

SETUP contamination, 290

signature, 285–286

unbalanced, 289

unconcealed messages, 285, 286

universal exponent, 288

running time, 253

S

Salomaa, Arto, 306

salting, 247, 248, 286

Schneier, Bruce, 245, 250, 306

SDA, see standard decoding array

Seberry, Jennifer, 306

Secretly Embedded Trapdoor with Universal Protection (SETUP), 290, 305

Secure Hash Algorithm (SHA), 283, 295

Seroussi, G., 305

SHA, see Secure Hash Algorithm

Shamir's 3-pass protocol, 298, 302

Shamir, Adi, 284, 302, 303

Shannon, C. E., 236

shift cipher, 231

shift registers, 173

Shimomura, Tsutomu, 245

shortened RS(2r, δ) code, 133

signature, 281

DSA, 295

DSS, 295

ECDSA, 305

ElGamal, 294–296

RSA, 285–286

with appendix, 282

with message recovery, 285
< previous page | page_349 | next page >


page_35

< previous page | page_35 | next page >

Page 35

2.3.18 Theorem 2.3.14 also holds in Rn. In Rn every vector can be written uniquely as the sum of a vector in 〈S〉 and a vector in S⊥, and the zero vector is the only vector 〈S〉 and S have in common. (For example, in R 3 take 〈S〉 to be the xy-plane and S⊥ the z-axis.) Use S = {000, 101} in K 3 to show that this is not the case in general in Kn.

The last result in this section deals with the question of how many different bases a linear code can have. In Rn a subspace has infinitely many bases, but this is not so in Kn.

Theorem 2.3.19 A linear code of dimension k has precisely [image] different bases.

Example 2.3.20 The linear code K 4 has dimension k = 4 and hence

different bases. Any linear code contained in Kn, for n ≥ 4, which has dimension 4 also has 840 different bases.

Exercises

2.3.21 Let bn be the number of different bases for Kn. Verify the entries in the following table:

n | 1 | 2 | 3 | 4 | 5 | 6
bn | 1 | 3 | 28 | 840 | 83,328 | 27,998,208

2.3.22 List all the bases for K2 and for K3.

2.3.23 Find the number of different bases for each code C = 〈S〉 for

(a) S = {010, 011, 111},

(b) S = {1010, 0101, 1111},

(c) S = {0101, 1010, 1100},

(d) S = {1000, 0100, 0010, 0001},

(e) S = {11000, 01111, 11110, 01010},

(f) S = {10101, 01010, 11111, 00011, 10110}.

2.4 Matrices

An m × n matrix is a rectangular array of scalars with m rows and n columns. We assume the reader is familiar with the algebra of matrices over the real numbers. In this section we review the necessary parts of elementary matrix theory needed for coding theory.
< previous page | page_35 | next page >


page_350

< previous page | page_350

Page 350

Simmons, Gustavus J., 249, 304, 306

Singleton bound, 67

Smart, Nigel P., 305

smooth integer, 270

Solovay-Strassen primality test, 266

square root, 271–272

algorithm, 272

relation to factoring, 272

SQ ROOT problem definition, 272

square-and-multiply algorithm, 258

standard decoding array (SDA), 59

standard form, 49

standard ordering of Km, 205

state cipher, 235

state diagram, 183

Stinson, Douglas R., 235, 249, 250, 283, 306

stream cipher, 235

subfield subcode, 127

subliminal channel, 304–305

substitution cipher, 230, 237

Sugarman, Robert, 250

symmetric difference, 213

symmetric-key encryption, 230

compared with public-key, 280

see also. cipher

syndrome, 57

syndrome polynomial, 103

systematic code, 49

T

tabular form, 183

Taylor, Bradley, 276, 306

Thompson, Eric, 245

transform, 141, 144

trapdoor one-way function, 281

truncated Viterbi decoding algorithm, 193

U

U.S. Army Signal Intelligence Service, 250

U.S. Army Signal Security Agency, 250

unconditional security, 229, 236

universal exponent, 288

V

van de Graaf, Jeroen, 302

van Oorshot, Paul C., 249, 306

Vanstone, Scott A., 249, 306

vector form, 142, 205

VENONA codebreak, 236, 250

Verheul, Eric R., 285

Vernam cipher, 236

Vigenère cipher, 232–235, 237

W

Wagner, David, 237

weight, 189

Wiener, Michael J., 243, 245, 286, 287

Williams' public-key scheme, 292

Williamson, Malcolm J., 305

window size, 188

Wright, Peter, 236

Y

Young, Adam, 290, 305

Yung, Moti, 290, 305

Z

zero-knowledge proof, 299–301, 306

based on QRP, 301

possession of a discrete log, 302

possession of a square root, 300

The Strange Cave of Ali Baba, 306

Where's Waldo, 306

Zimmermann, Philip R., 290, 306
< previous page | page_350


page_36

< previous page | page_36 | next page >

Page 36

If A is an m × n matrix and B is an n × p matrix, then the product AB is the m × p matrix which has for its (i, j)th entry (that is, the entry in row i and column j) the dot product of row i of A and column j of B. For example

Note that the number of columns of the first matrix must equal the number of rows of the second matrix in order for the product to be defined.

Exercises

2.4.1 Find the product of each pair of the following matrices whenever the product is defined.

The usual algebraic rules for matrices over the reals also hold for matrices over K. The m × n zero matrix is the m × n matrix with each entry equal to 0. The n × n (square) matrix I in which the (i, j)th entry is 1 if i = j and is 0 otherwise is the n × n identity matrix. For any matrix A, AI = A and IA = A. The next three exercises point out three algebraic rules which fail for matrices over K.

Exercises

2.4.2 Find 2 × 2 matrices A and B over K such that AB ≠ BA.

2.4.3 Find 2 × 2 matrices A and B over K, both different from the zero matrix 0, such that AB = 0.

2.4.4 Find 2 × 2 matrices A, B, and C over K such that AB = AC but B ≠ C.

There are two types of elementary row operations which may be performed on a matrix over K. They are:

1. interchanging two rows, and

2. replacing a row by itself plus another row.

Two matrices are row equivalent if one can be obtained from the other by a sequence of elementary row operators.

A 1 in a matrix M (over K) is called a leading 1 if there are no 1s to its left in the same row, and a column of M is called a leading column if it contains a leading 1. M is in row echelon form (REF) if the zero rows of M (if any) are all at

< previous page | page_36 | next page >


page_37

< previous page | page_37 | next page >

Page 37

the bottom, and each leading 1 is to the right of the leading 1s in the rows above. If further, each leading column contains exactly one 1, M is in reduced row echelon form (RREF).

Any matrix over K can be put in REF or RREF by a sequence of elementary row operations. In other words, a matrix is row equivalent to a matrix in REF or in RREF. For a given matrix, its RREF is unique, but it may have many REFs.

Example 2.4.5 We find the RREF for the matrix M below using elementary row operations

Exercises

2.4.6 Find the RREF for each of the four matrices in Exercise 2.4.1.

The transpose of an m × n matrix A is the n × m matrix AT which has column i of A as its i-th row. For example,

We will need two facts about the transpose of matrices A and B: (AT)T = A and (AB)T = BT AT.

2.5 Bases for C = 〈S〉 and C⊥

We develop algorithms for finding bases for a linear code and its dual. These methods will be of great assistance in our study of linear codes.

Let S be a nonempty subset of Kn. The first two algorithms provide a basis for C = 〈S〉, the linear code generated by S.

Algorithm 2.5.1 Form the matrix A whose rows are the words in S. Use elementary row operations to find a REF of A. Then the nonzero rows of the REF form a basis for C = 〈S〉.
< previous page | page_37 | next page >


page_38

< previous page | page_38 | next page >

Page 38

The algorithm works because the rows of A generate C and elementary row operations simply interchange words or replace one word (row) with another in C giving a new set of codewords which still generates C. Clearly the nonzero rows of a matrix in REF are linearly independent.

Example 2.5.2 We find a basis for the linear code C = 〈S〉 for S = {11101, 10110, 01011, 11010}.

The last matrix is a REF of A. By Algorithm 2.5.1, {11101, 01011, 00111} is a basis for C = 〈S〉. Another REF of A is

so {11101, 01100, 00111} is also a basis for C = 〈S〉. Note that Algorithm 2.5.1 does not produce a unique basis for 〈S〉, nor are the words in the basis necessarily in the given set S.

Exercises

2.5.3 Use Algorithm 2.5.1 to find a basis for C = 〈S〉 for each of the following sets S.

(a) S = {010, 011, 111}

(b) S = {1010, 0101, 1111}

(c) S = {0101, 1010, 1100}

(d) S = {1000, 0100, 0010, 0001}

(e) S = {11000, 01111, 11110, 01010}

(f) S = {10101, 01010, 11111, 00011, 10110}

(g) S = {0110, 1010, 1100, 0011, 1111}

(h) S = {111000, 000111, 101010, 010101}

(i) S = {00000000, 10101010, 01010101, 11111111}

Algorithm 2.5.4 (finding a basis for C) Form the matrix A whose columns are the words in S. Use elementary row operations to place A in REF and locate the leading columns in the REF. Then the original columns of A corresponding to these leading columns form a basis for C = 〈S〉.

It is shown in elementary linear algebra that a linearly independent set of columns of a matrix is still linearly independent after applying a sequence of elementary row operations to the matrix. It is easy to see that the leading columns of a matrix in REF form a linearly independent set.
< previous page | page_38 | next page >


page_39

< previous page | page_39 | next page >

Page 39

Example 2.5.5 We use Algorithm 2.5.4 to find a basis for C = 〈S〉 for the set S of Example 2.5.2.

Since columns 1, 2, and 4 of the REF are the leading columns, Algorithm 2.5.4 says that columns 1, 2, and 4 of A form a basis for C = 〈S〉. This basis {11101, 10110, 11010}. Note that Algorithm 2.5.4 has the property of producing a basis for C = 〈S〉, all of whose elements are words in the given set S.

Exercises

2.5.6 Use Algorithm 2.5.4 to find a basis for C = 〈S〉 for each set S in Exercise 2.5.3 and compare answers.

Now we give an algorithm for finding a basis for the dual code C⊥. It will be a very useful algorithm in our subsequent work. Also, notice that this algorithm provides a basis for C (since it includes Algorithm 2.5.1).

Algorithm 2.5.7 (finding a basis for C⊥) Form the matrix A whose rows are the words in S. Use elementary row operations to place A in RREF. Let G be the k × n matrix consisting of all the nonzero rows of the RREF. Let X be the k × (n – k) matrix obtained from G by deleting the leading columns of G. Form an n ×(n – k) matrix H as follows:

(i) in the rows of H corresponding to the leading columns of G, place, in order, the rows of X;

(ii) in the remaining n – k rows of H, place, in order, the rows of the (n − k) × (n – k) identity matrix I.

Then the columns of H form a basis for C⊥.

The algorithm works because the n − k columns of H are linearly independent, dim C⊥ = n − dim C = n − k, and to within a permutation of the columns of G and the rows of H, GH = X + X = 0.

The following description of Algorithm 2.5.7 may help in remembering it. The matrix G contains k leading columns. Permute the columns of G so that these columns come first. The other columns form the matrix X. Call this matrix G′. Then Algorithm 2.5.7 begins thus (each entry in O is 0):
< previous page | page_39 | next page >


page_4

< previous page | page_4 | next page >

Page 4

A binary channel is symmetric if 0 and 1 are transmitted with equal accuracy; that is the probability of receiving the correct digit is independent of which digit, 0 or 1, is being transmitted. The reliability of a binary symmetric channel (BSC) is a real number p, 0 ≤ p ≤ 1, where p is the probability that the digit sent is the digit received.

If p is the probability that the digit received is the same as the digit sent, then 1 − p is the probability that the digit received is not the digit sent. The following diagram may clarify how a BSC operates:

In most cases it may be hard to estimate the actual value of p for a given channel. However the actual value of p does not influence significantly the form of the theory.

We will call one channel more reliable than another if its reliability is higher. Note that if p = 1, then there is no chance of a digit being altered in transmission. Hence the channel is perfect and of no interest to us. Nor is a channel with p = 0 of any interest. Any channel with 0 < p ≤ 1/2 can be easily converted into a channel with 1/2 ≤ p < 1. Henceforth we will always assume that we are using a BSC with probability p satisfying 1/2 < p < 1. (The case where p = 1/2 is raised in Exercise 1.2.6.)

Exercises

1.2.4 Explain why a channel with p = 0 is uninteresting.

1.2.5 Explain how to convert a channel with 0 < p ≤ 1/2 into a channel with 1/2 ≤ p < 1.

1.2.6 What can be said about a channel with p = 1/2?

1.3 Correcting and detecting error patterns

We consider now the possibilities of correcting and detecting errors. In this section we intend to develop an intuitive understanding of the concepts involved in correcting and detecting errors, while a formal approach is adopted in later sections.

Suppose a word is received that is not a codeword. Clearly some error has occurred during the transmission process, so we have detected that an error (perhaps several errors) has occurred. If however a codeword is received, then perhaps no errors occurred during transmission, so we cannot detect any error.
< previous page | page_4 | next page >


page_40

< previous page | page_40 | next page >

Page 40

Permute the columns of G to form G′ = [Ik, X]. Form a matrix H′ as follows:

Apply the inverse of the permutation applied to the columns of G to the rows of H′ to form H.

Example 2.5.8 We use Algorithm 2.5.7 to find a basis for C⊥ for the set S of Example 2.5.2.

which is in RREF. Now [image]. The leading columns of G are columns 1, 2, and 3, so the rows of X are placed in rows 1, 2, and 3 respectively, of the 5 × (5 – 3) matrix H. The remaining rows of H are filled with the 2 × 2 identity matrix. Thus

By Algorithm 2.5.7, the columns of H form a basis for C⊥. Note that, by Algorithm 2.5.1, the rows of G form a basis for C = 〈S〉.

Example 2.5.9 Suppose n = 10 and we have a set S of words in K 10. Suppose the RREF of the matrix A in Algorithm 2.5.7 has nonzero rows

The leading columns of G are columns 1, 4, 5, 7, and 9. We permute the columns of G into the order 1, 4, 5, 7, 9, 2, 3, 6, 8, 10 (so the leading columns are first) to form the matrix
< previous page | page_40 | next page >


page_41

< previous page | page_41 | next page >

Page 41

Then we form the matrix H′ and finally rearrange the rows of H into their natural order to form the matrix H′.

By Algorithm 2.5.7, the columns of H form a basis for C⊥.

Exercises

2.5.10 Use Algorithm 2.5.7 to find a basis for C⊥ for each of the codes C = 〈S〉 where

(a) S = {010, 011, 111}

(b) S = {1010, 0101, 1111}

(c) S = {0101, 1010, 1100}

(d) S = {1000, 0100, 0010, 0001}

(e) S = {11000, 01111, 11110, 01010}

(f) S = {10101, 01010, 11111, 00011, 10110}

(g) S = {0110, 1010, 1100, 0011, 1111}

(h) S = {111000, 000111, 101010, 010101}

(i) S = {00000000, 10101010, 01010101, 11111111}.

2.5.11 With the notation of Algorithm 2.5.7, explain why it is expected that GH = 0.

2.5.12 For each of the following sets S, use Algorithm 2.5.7 to produce a basis B for the code C = 〈S〉 and a basis B⊥ for the dual code C⊥.

(a) S = {000000, 111000, 000111, 111111}

(b) S = {1101000, 0110100, 0011010, 0001101, 1000110, 0100011, 1010001}

(c) S = {1111000, 0111100, 0011110, 0001111, 1000111, 1100011, 1110001}

(d) S = {101101110, 011011101, 110110010, 011011110, 111111101}

(e) S = {100100100, 010010010, 11111111, 000000000}

(f) S = {001101, 001000, 001111, 000101, 000001}

2.6 Generating matrices and encoding

We put the material of the last several sections to work to find an important matrix for a linear code and to see how this matrix is used to transmit messages.
< previous page | page_41 | next page >


page_42

< previous page | page_42 | next page >

Page 42

First a few preliminary notes. The rank of a matrix over K is the number of nonzero rows in any REF of the matrix. The dimension k of the code C is the dimension of C, as a subspace of kn. If C also has length n and distance d, then we refer to C as an (n, k, d) linear code. These three parameters, length, dimension and distance, provide vital information about C.

If C is a linear code of length n and dimension k, then any matrix whose rows form a basis for C is called a generator matrix for C. Note that a generator matrix for C must have k rows and n columns, and it must have rank k.

Theorem 2.6.1 A matrix G is a generator matrix for some linear code C if and only if the rows of G are linearly independent; that is, if and only if the rank of G is equal to the number of rows of G.

Because row equivalent matrices have the same rank, we have the following theorem.

Theorem 2.6.2 If G is a generator matrix for a linear code C, then any matrix row equivalent to G is also a generator matrix for C. In particular, any linear code has a generator matrix in RREF.

To find a generator matrix for a linear code C, form the matrix whose rows are the words in C. Since C = 〈C〉, either Algorithm 2.5.1 or Algorithm 2.5.7 can be used to produce a basis for C. The matrix whose rows are these basis vectors is a generator matrix for C.

Example 2.6.3 We find a generator matrix for the code C = {0000, 1110, 0111, 1001}. Using Algorithm 2.5.1,

so [image] is a generator matrix for C. By Algorithm 2.5.7, since the RREF

of A is [image] [image] is also a generator matrix for C.

Exercises

2.6.4 Determine whether each of the following is a generator matrix for some

< previous page | page_42 | next page >


page_43

< previous page | page_43 | next page >

Page 43

linear code.

2.6.5 Find a generator matrix in RREF for each of the following codes.

(a) C = {000, 001, 010, 011}

(b) C = {0000, 1001, 0110, 1111}

(c) C = {00000, 11111}

(d) C = {00000, 11100, 00111, 11011}

(e) C = {00000, 11110, 01111, 10001}

(f) C = {000000, 101010, 010101, 111111}

2.6.6 Find a generator matrix for each of the following codes. Give the dimension of the code.

(a) C = {000000, 001011, 010101, 011110, 100110, 101101, 110011, 111000}

(b) C = {00000000, 01101111, 11011000, 11111101, 10010010, 00100101, 01001010, 10110111}

(c) C = {0000000000, 1111100000, 0000011111, 1111111111}

2.6.7 Find a generator matrix for the linear code generated by each of the following sets. Give the parameters (n,k,d) for each code.

(a) S = {11111111, 11110000, 11001100, 10101010}

(b) S = {11111100, 11110011, 11001111, 00111111}

(c) S = {100100100, 010010010, 001001001, 111111111}

(d) S = {10101, 01010, 11111, 00011, 10110}

(e) S = {1010, 0101, 1111}

(f) S = {101101, 011010, 110111, 000111, 110000}

(g) S = {1001011, 0101010, 1001100, 0011001, 0000111}

Let C be a linear code of length n and dimension k. If G is a generator matrix for C and if u is a word of length k written as a row vector, then v = uG is a word in C, since v is a linear combination of the rows of G, which form a basis for C. Indeed, if u = (a1, a2,...,ak) and if

where g1, g2,...,gk are the rows of G, then v = uG = a1 g1 + a2 g2 +...+ ak gk. On the other hand, since every word v in C is a linear combination of basis words

< previous page | page_43 | next page >


page_44

< previous page | page_44 | next page >

Page 44

(rows of G), then v = uG for some u in Kk. Moreover, if u1 G = u2 G, then u1 = u2 since each word in C is a unique linear combination of the words in a basis. Thus no word v = uG is produced by more than one u in Kk.

Theorem 2.6.8 If G is a generator matrix for a linear code C of length n and dimension k, then v = uG ranges over all 2k words in C as u ranges over all 2k words of length k. Thus C is the set of all words uG, u in Kk. Moreover, u1 G = u2 G if and only if u1 = u2.

Note that Theorem 2.6.8 says that the messages that can be encoded by a linear (n,k,d) code are exactly all messages u in Kk. The message u is encoded as v = uG, so only k digits in any codeword are used to carry the message. Notice that the information rate of an (n,k,d) code is log2(2k)/n = k/n.

Example 2.6.9 Let C be the (5, 3, d) linear code with generator matrix shown below. The information rate of C is k/n = 3/5. All messages u in K 3 may be encoded. For example the message u = 101 is encoded as

Exercises

2.6.10 For each of the following generating matrices, encode the given messages.

2.6.11 Assign messages to the words in K 3 as follows:

000 | 100 | 010 | 001 | 110 | 101 | 011 | 111
A | B | E | H | M | R | T | W

Using the generator matrix in Example 2.6.9, encode the message BE THERE (ignore the space).
< previous page | page_44 | next page >


page_45

< previous page | page_45 | next page >

Page 45

2.6.12 Let C be the code with generator matrix

Assign messages to the words in K 4 as follows:

0000 | 1000 | 0100 | 0010 | 0001 | 1100 | 1010 | 1001
A | B | C | D | E | F | G | H
0110 | 0101 | 0011 | 1110 | 1101 | 1011 | 0111 | 1111
I | J | K | L | M | N | O | P

(a) Encode the message HELP.

(b) Transmit the message HELP assuming that during transmission the first word receives an error in the first position, the second word receives no errors, the third an error in the seventh position, and the fourth an error in the fifth and sixth positions.

(c) Encode the message CALL HOME BAMA (ignore the spaces).

2.6.13 Find the number of messages which can be sent, and the information rate r, for each of the linear codes in Exercises 2.6.6 and 2.6.7.

2.7 Parity-check matrices

We develop another matrix associated with a linear code and closely connected with the generator matrix. This new matrix will be of great value in designing decoding schemes.

A matrix H is called a parity-check matrix for a linear code C if the columns of H form a basis for the dual code C⊥. If C has length n and dimension k, then, since the sum of the dimensions of C and C⊥ is n, any parity-check matrix for C must have n rows, n – k columns and rank n – k. Compare the following theorem to Theorem 2.6.1.

Theorem 2.7.1 A matrix H is a parity-check matrix for some linear code C if and only if the columns of H are linearly independent.

The next theorem describes a linear code in terms of its parity-check matrix.

Theorem 2.7.2 If H is a parity-check matrix for a linear code C of length n, then C consists precisely of all words v in Kn such that vH = 0.

If we are given a generator matrix for a linear code C, then we can find a parity-check matrix for C using Algorithm 2.5.7. The parity-check matrix is the matrix H constructed in Algorithm 2.5.7, since the columns of H form a basis for C⊥.
< previous page | page_45 | next page >


page_46

< previous page | page_46 | next page >

Page 46

Example 2.7.3 We find a parity-check matrix for the code C = {0000, 1110, 0111, 1001} of Example 2.6.3. There we found that

is a generator matrix for C which is in RREF. By Algorithm 2.5.7, we construct H:

is a parity-check matrix for C. Note that vH = 00 for all words v in C.

Exercises

2.7.4 Find a parity-check matrix from each of the following codes.

(a) C = {000, 001, 010, 011}

(b) C = {0000, 1001, 0110, 1111}

(c) C = {00000, 11111}

(d) C = {00000, 11100, 00111, 11011}

(e) C = {00000, 11110, 01111, 10001}

(f) C = {000000, 101010, 010101, 111111}

2.7.5 Find a parity-check matrix for each of the following codes (the generating matrices were constructed in Exercises 2.6.6 and 2.6.7).

(a) C = {000000, 001011, 010101, 011110, 100110, 101101, 110011, 111000}

(b) C = {00000000, 01101111, 11011000, 11111101, 10010010, 00100101, 01001010, 10110111}

(c) C = {0000000000, 1111100000, 0000011111, 1111111111}

(d) C = 〈S〉, S = {11111111, 11110000, 11001100, 10101010}

(e) C = 〈S〉, S = {11111100, 11110011, 11001111, 00111111}

(f) C = 〈S〉, S = {100100100, 010010010, 001001001, 111111111}

(g) C = 〈S〉, S = {10101, 01010, 11111, 00011, 10110}

(h) C = 〈S〉, S = {1010, 0101, 1111}

(i) C = 〈S〉, S = {101101, 011010, 110111, 000111, 110000}

(j) C = 〈S〉, S = {1001011, 0101010, 1001100, 0011001, 0000111}

We now characterize the relationship between a generator matrix and parity check matrix for a linear code, and the relationship between these matrices for a linear code and its dual code.

Theorem 2.7.6 Matrices G and H are generating and parity-check matrices, respectively, for some linear code C if and only if
< previous page | page_46 | next page >


page_47

< previous page | page_47 | next page >

Page 47

(i) the rows of G are linearly independent,

(ii) the columns of H are linearly independent,

(iii) the number of rows of G plus the number of columns of H equals the number of columns of G which equals the number of rows of H, and

(iv) GH = 0.

Theorem 2.7.7 H is a parity-check matrix of C if and only if HT is a generator matrix for C⊥.

Theorem 2.7.7 follows from Theorem 2.7.6 and the fact that

HT GT = (GH)T = 0.

Given any one of the generating or parity-check matrices of C or of C⊥, Algorithm 2.5.7 and Theorem 2.7.7 can be used to form the other three matrices. The following diagram indicates how this is done.

Example 2.7.8 Let C be a linear code with parity-check matrix

(a) Then a generator matrix for C⊥ is

(b) The RREF of HT is [image], so, by Algorithm 2.5.7, a parity-check matrix for C⊥ is
< previous page | page_47 | next page >


page_48

< previous page | page_48 | next page >

Page 48

(c) From the form of H, we have that

is a generator matrix for C. This is seen by using Algorithm 2.5.7 backwards. Thus, by Theorem 2.7.7 GT is also a parity-check matrix for C⊥.

Exercises

2.7.9 In each part, a parity-check matrix for a linear code C is given. Find (i) a generator matrix for C⊥; (ii) a generator matrix for C.

2.7.10 List all the words in the dual code C⊥ for the code C = {00000, 11111}. Then find generating and parity-check matrices for C⊥

2.7.11 For each code C described below, find the dimension of C, the dimension of C⊥, the size of generating and parity-check matrices for C and for C⊥, the number of words in C and in C⊥, and the information rates r of C and C⊥.

(a) C has length n = 2t – 1 and dimension t.

(b) C has length n = 23 and dimension 11.

(c) C has length n = 15 and dimension 8.

2.8 Equivalent codes

Any k × n matrix G with k < n whose first k columns form the k × k identity matrix Ik, so

G = [Ik, X],

automatically has linearly independent rows and is in RREF. Thus G is a generator matrix for some linear code of length n and dimension k. Such a generator matrix

< previous page | page_48 | next page >


page_49

< previous page | page_49 | next page >

Page 49

is said to be in standard form, and the code C generated by G is called a systematic code.

Not all linear codes have a generator matrix in standard form. For example, the code defined by the generator matrix in the exercise below has five other generating matrices; none of them are in standard form, and neither is G.

Exercises

2.8.1 Find the other five generator matrices for the code generated by

It is desirable, however, to use codes having generating matrices in standard form. One reason for this is that if a linear code C has generator matrix G in standard form, G = [I, X], then Algorithm 2.5.7 yields at once that

is a parity-check matrix for C.

By Theorem 2.6.8 each codeword v in a linear code C of length n and dimension k is equal to uG for one and only word u in Kk, where G is a generator matrix for C. We think of the word u of length k as the message to be sent. But rather than transmitting u, we of course transmit the codeword v = uG. If MLD manages to conclude correctly that v = uG was sent, then the recipient of the transmission must recover somehow the original message u from uG. If G is in standard form, then it is trivial to recover u from uG. For in this case

v = uG = u[I X] = [uI uX] = [u uX].

So we obtain the following theorem, which points out an important advantage of having a generator matrix in standard form.

Theorem 2.8.2 If C is a linear code of length n and dimension k with generator matrix G in standard form, then the first k digits in the codeword v = uG form the word u in Kk.

Example 2.8.3 If

and if the message is u = 0111, then uG = 0111001 = [u001]. And if u = 1011, then uG = 1011000.
< previous page | page_49 | next page >


page_5

< previous page | page_5 | next page >

Page 5

The concept of correcting an error is more involved. As in the introduction when we were inclined to correct 'gub' to 'gun' rather than to 'rat', we appeal to intuition to suggest that any received word should be corrected to a codeword that requires as few changes as possible. (In a later section we show that the probability that such a codeword was sent is at least as great as the probability that any other codeword was sent.) To consolidate these ideas, we shall discuss some particular codes. Notice that our assumption that no digits are lost or created in transmission precludes decoding 'gub' to 'firetruck'.

Example 1.3.1 Let C 1 = {00, 01, 10, 11}. Every received word is a codeword and so C 1 cannot detect any errors. Also C 1 corrects no errors since every received word requires no changes to become a codeword.

Example 1.3.2 Modify C 1 by repeating each codeword three times. The new code is

C 2 = {000000, 010101, 101010, 111111}.

This is an example of a repetition code. Suppose that 110101 is received. Since this is not a codeword we can detect that at least one error has occurred. The codeword 010101 can be formed by changing one digit, but all other codewords are formed by changing more than one digit. Therefore we expect that 010101 was the most likely codeword transmitted, so we correct 110101 to 010101. (A codeword that can be formed from a word w with the least number of digits being changed is called a closest codeword; this idea is formalized later.) In fact if any of the codewords, c ε C 2, is transmitted and one error occurs during transmission, then the unique closest codeword to the received word is c; so any single error results in a word that we correct to the codeword that was transmitted.

Example 1.3.3 Modify C 1 by adding a third digit to each codeword so that the number of 1's in each codeword is even. The resulting code is

C 3 = {000, 011, 101, 110}.

The added digit is called a parity-check digit. Suppose 010 is received, then since 010 is not a codeword we can detect that an error has occurred. Each of the codewords 110, 000 and 011 can be formed by changing one digit in the received word. In later sections we distinguish between how we treat received words that are closest to a unique codeword (and so is the single most likely codeword sent) as was the case in Example 1.3.2, and received words that are closest to several codewords as in this example. It suffices at this stage to observe that it seems more sensible to correct 010 to one of 110, 000 or 011 rather than to 101.

Exercises

1.3.4 Let C be the code of all words of length 3. Determine which codeword was most likely sent if 001 is received.
< previous page | page_5 | next page >


page_50

< previous page | page_50 | next page >

Page 50

Exercises

2.8.4 Let C be the generator matrix in Example 2.8.3. Encode each of the following messages u, and observe that the first 4 digits in the resulting codeword form the message u.

(a) u = 1111 (b) u = 1011 (c) u = 0000

2.8.5 Explain a method for recovering u from uG if G is not in standard form.

2.8.6 If a linear code C has generator matrix

recover u from v = uG = 0000101.

Under the hypotheses of Theorem 2.8.2 the first k digits of the codeword v = uG are called the information digits, since they actually contain the message u, while the last n – k digits of v = uG are called the redundancy or parity-check digits.

With all these advantages of having a linear code with generator matrix in standard form, what can be done if we are stuck with a code C having no generator matrix in standard form? Consider the code C with generator matrix G from Exercise 2.8.1 (see below). As indicated in Exercise 2.8.1, C has no generator matrix in standard form. Suppose, in this example, we decide to rearrange the digits in all codewords and transmit the digits in the order ''first, third, second," rather than "first, second, third." The four words in C have been transformed, then, into the four words in the new code C′ as indicated in the following chart:

C = {000, 100, 001, 101}

C′ = {000, 100, 010, 110}

Note that C′, although a different code from C, shares many properties with C. For example, both C and C′ are linear; both have length 3, dimension 2 and distance 1. But C′ has an advantage over C, namely C′ has a generator matrix in standard form. Observe that G′ is obtained from G by switching the second and third columns, just as C′ is obtained from C by consistently switching the second and third digits.
< previous page | page_50 | next page >


page_51

< previous page | page_51 | next page >

Page 51

If C is any block code of length n, we can always obtain a new block code C′ of length n by choosing a particular permutation of the n digits and then consistently rearranging every word in C in the chosen way. The resulting code C′ is said to be equivalent to C.

Example 2.8.7 If n = 5 and we choose to rearrange the digits in the order 2, 1, 4, 5, 3, then the code

C = {11111, 01111, 00111, 00011, 00001}

is equivalent to the code

C′ = {11111, 10111, 00111, 00110, 00010}.

(Note that C and C′ are not linear.)

Theorem 2.8.8 Any linear code C is equivalent to a linear code C′ having a generator matrix in standard form.

Proof: If G is a generator matrix for C, place G in RREF. Rearrange the columns of the RREF so that the leading columns come first and form an identity matrix. The result is a matrix G′ in standard form which is a generator matrix for a code C′ equivalent to C.

Example 2.8.9 The matrix

is a generator matrix in RREF with columns 2, 4, 5, 6, and 9 as leading columns. Rearranging the columns in the order 2, 4, 5, 6, 9, 1, 3, 7, 8 yields the matrix

which is a generator matrix in standard form for a code equivalent to the code generated by G.

Exercises

2.8.10 Find a systematic code C′ equivalent to the given code C. Check that C and C′ have the same length, dimension, and distance.
< previous page | page_51 | next page >


page_52

< previous page | page_52 | next page >

Page 52

(a) C = {00000, 10110, 10101, 00011}

(b) C = {00000, 11100, 00111, 11011}.

2.8.11 Find a generator matrix G in standard form for a code equivalent to the code with given generator matrix G.

2.8.12 Find a generator matrix G′ in standard form for a code C′ equivalent to the code C with given parity-check matrix H.

2.8.13 Prove that equivalent linear codes always have the same length, dimension, and distance.

2.8.14 Determine whether each of the following pairs of matrices G 1 and G 2 generate equivalent codes.

2.9 Distance of a linear code

We observed that the distance of a linear code is the minimum weight of any nonzero codeword. The distance of a linear code can also be determined from a parity-check matrix for the code.
< previous page | page_52 | next page >


page_53

< previous page | page_53 | next page >

Page 53

Theorem 2.9.1 Let H be a parity-check matrix for a linear code C. Then C has distance d if and only if any set of d – 1 rows of H is linearly independent, and at least one set of d rows of H is linearly dependent.

The idea is that if v is a word, then vH is a linear combination of exactly wt(v) rows of H. So if v is in C and wt(v) = d, then since vH = 0, some d rows of H are linearly dependent. And if vH = 0 then v is a codeword so wt(v) ≥ d.

Example 2.9.2 Let C be the linear code with parity-check matrix

By inspection it is seen that no two rows of H sum to 000, so any two rows of H are linearly independent, but rows 1, 3, and 4, for instance sum to 000, and hence are linearly dependent. Therefore d – 1 = 2, so the distance of C is d = 3.

Exercises

2.9.3 Find the code C in the Example 2.9.2. Compute the weight of each codeword and verify that C has distance 3.

2.9.4 Find the distance of the linear code C with each of the given parity-check matrices. Use Theorem 2.9.1 and then check your answer by finding wt(v) for each v in C.

2.9.5 Find, by Theorem 2.9.1, the distance of the linear code with the given generator matrix.

2.10 Cosets

In this section we consider a topic which will be useful in decoding a linear code, to which we will turn in the next section.
< previous page | page_53 | next page >


page_54

< previous page | page_54 | next page >

Page 54

If C is a linear code of length n, and if u is any word of length n, we define the coset of C determined by u to be the set of all words of the form v + u as v ranges over all the words in C. We denote this coset by C + u. Thus

C + u = {v + u | v ∈ C}.

Example 2.10.1 Let C = {000, 111}, and let u = 101. Then

C + 101 = {000 + 101, 111 + 101} = {101, 010}.

Note that also

C + 111 = {000 + 111, 111 + 111} = {111, 000} = C

and

C + 010 = {000 + 010, 111 + 010} = {010, 101} = C + 101.

Exercises

2.10.2 List the rest of the cosets of C = {000, 111}. Notice that there are eight possibilities for the cosets of C, one for each word in K 3, but only four of these cosets are distinct.

If C is a linear code of length n, then you might think that there are as many as 2n different cosets C + u of C, one for each of the 2n different words u of length n. As Example 2.10.1 and Exercise 2.10.2 show, this is almost never so. It is quite possible for C + u 1 to be identical with C + u 2, but yet u 1 ≠ u 2.

The following theorem contains several important and useful facts about cosets. A careful study of the examples following the theorem should help in understanding these facts. The proofs are technical, set-theoretic arguments, and hence relegated to the exercises.

Theorem 2.10.3 Let C be a linear code of length n. Let u and v be words of length of n.

1. If u is in the coset C + v, then C + u = C + v; that is, each word in a coset determines that coset.

2. The word u is in the coset C + u.

3. If u + v is in C, then u and v are in the same coset.

4. If u + v is not in C, then u and v are in different cosets.

5. Every word in Kn is contained in one and only one coset of C; that is, either C + u = C + v, or C + u and C + v have no words in common.

6. |C + u| = |C| that is, the number of words in a coset of C is equal to the number of words in the code C.
< previous page | page_54 | next page >


page_55

< previous page | page_55 | next page >

Page 55

7. If C has dimension k, then there are exactly 2n−k different cosets of C, and each coset contains exactly 2k words.

8. The code C itself is one of its cosets.

Example 2.10.4 We list the cosets of the code

C = {0000, 1011, 0101, 1110}.

First of all, C itself is a coset by (8) of Theorem 2.10.3. (Numbers in parentheses suggest that the reader refer to the parts of Theorem 2.10.3.) Every word in C will determine the coset C (by (1) and (5)), so we pick a word u in K 4 not in C. For later use in decoding, it will help to pick u of smallest weight possible. So let's take u = 1000. Then we get the coset

C + 1000 = {1000, 0011, 1101, 0110}

by adding 1000 to each word in C. Note that u = 1000 is in the coset C + u = C + 1000. Now pick another word, of small weight, in K 4 but not in C or C + 1000, say 0100. Form another coset

C + 0100 = {0100, 1111, 0001, 1010}.

Repeating the process with 0010 yields the coset

C + 0010 = {0010, 1001, 0111, 1100}.

The code C has dimension k = 2. We have listed 2n–k = 24–2 = 22 = 4 cosets, each with 2k = 22 = 4 words, and every word in K 4 is accounted for by appearing in exactly one coset. Also observe that 0001 + 1010 = 1011 is in C, thus 0001 and 1010 are in the same coset, namely C + 0100 (see (3)). On the other hand, 0100 + 0010 = 0110 is not in C, and 0100 and 0010 are in different cosets (see (4)).

Example 2.10.5 We list the cosets of the linear code C with the generator matrix

000000 | 100000 | 010000 | 001000
100110 | 000110 | 110110 | 101110
010011 | 110011 | 000011 | 011011
001111 | 101111 | 011111 | 000111
110101 | 010101 | 100101 | 111101
101001 | 001001 | 111001 | 100001
011100 | 111100 | 001100 | 010100
111010 | 011010 | 101010 | 110010
< previous page | page_55 | next page >


page_56

< previous page | page_56 | next page >

Page 56

000100 | 000010 | 000001 | 000101
100010 | 100100 | 100111 | 100011
010111 | 010001 | 010011 | 010110
001011 | 001101 | 001110 | 001010
110001 | 110111 | 110100 | 110000
101101 | 101011 | 101000 | 101100
011000 | 011110 | 011101 | 011001
111110 | 111000 | 111000 | 111111

The eight cosets are listed. The first is the code C itself. The word u used to form C + u is the top word in each coset, since u = 0 + u, and was chosen as in Example 2.10.4.

Exercises

2.10.6 List the cosets of each of the following linear codes.

(a) C = {0000, 1001, 0101, 1100}

(b) C = {0000, 1010, 1101, 0111}

(c) C = {00000, 10100, 01011, 11111}

(d) C = {0000}.

2.10.7 List the cosets of each of the linear codes having the given generator matrix.

2.10.8 List the cosets of the code having the given parity-check matrix.

2.10.9 Prove Theorem 2.10.3.
< previous page | page_56 | next page >


page_57

< previous page | page_57 | next page >

Page 57

2.11 MLD for linear codes

One of our goals is to design codes which permit easy and rapid decoding of a received word. Linear codes do in fact admit a more efficient method for implementing MLD than using an IMLD table. We will describe a procedure for either CMLD or IMLD for a linear code. The parity-check matrix and the cosets of the code play fundamental roles in the decoding process.

Let C be a linear code. Assume the codeword v in C is transmitted and the word w is received, resulting in the error pattern u = v + w. Then w + u = v is in C, so the error pattern u and the received word w are in the same coset of C by (3) of Theorem 2.10.3.

Since error patterns of small weight are the most likely to occur, here is how MLD works for a linear code C. Upon receiving the word w, we choose a word u of least weight in the coset C + w (which must contain w) and conclude that v = w + u was the word sent.

Example 2.11.1 Let C = {0000, 1011, 0101, 1110}. The cosets of C (Example 2.10.4) are

0000 | 1000 | 0100 | 0010
1011 | 0011 | 1111 | 1001
0101 | 1101 | 0001 | 0111
1110 | 0110 | 1010 | 1100

Suppose w = 1101 is received. The coset C + w = C + 1101 containing w is the second one listed. The word of least weight in this coset is u = 1000, which we choose as the error pattern. We conclude that v = w + u = 1101 + 1000 = 0101 was the most likely codeword sent. Now suppose w = 1111 is received. In the coset C + w containing 1111 there are two words of smallest weight, 0100 and 0001. Since we are doing CMLD, we arbitrarily select one of these, say u = 0100, for the error pattern, and conclude that v = w + u = 1111 + 0100 = 1011 was a most likely codeword sent.

Exercises

2.11.2 Let C be the code of Example 2.10.5. Use the procedure for CMLD just outlined to decode each of the following received words.

(a) 000011

(b) 001001

(c) 001101

(d) 010110

(e) 110101

(f) 001010

The hardest parts of the above procedure are searching to find the coset containing the received word w and then finding a word of least weight in that coset. We can use a parity-check matrix to develop a slick procedure for easing these burdens.

Let C be a linear code of length n and dimension k. Let H be a parity check matrix for C. For any word w in Kn, the syndrome of w is the word wH in Kn–k.
< previous page | page_57 | next page >


page_58

< previous page | page_58 | next page >

Page 58

Example 2.11.3 For the code C of Example 2.11.1 above, the matrix H below is a parity-check matrix. If w = 1101, then the syndrome of w is

Notice that the word of least weight in the coset C + w is u = 1000 (see Example 2.11.1), and the syndrome of u is

Furthermore, if w = 1101 is received, CMLD concludes v = w + u = 1101 + 1000 = 0101 was sent, so there was an error in the first digit. Notice also that for the error pattern u, the syndrome uH picks up the row of H, the first, corresponding to the location of the most likely error.

The following theorem contains some basic and useful facts about the syndrome. Proofs may be constructed using the definitions of the concepts involved and the properties of cosets from Theorem 2.10.3.

Theorem 2.11.4 Let C be a linear code of length n. Let H be a parity-check matrix for C. Let w and u be words in Kn.

1. wH = 0 if and only if w is a codeword in C.

2. wH = uH if and only if w and u lie in the same coset of C.

3. If u is the error pattern in a received word w, then uH is the sum of the rows of H that correspond to the positions in which errors occurred in transmission.

Note that if no errors occur in transmission and w is received, then wH = 0. But wH = 0 does not imply that no errors occurred, since the codeword w need not be the codeword that was sent.

Since words in the same coset have the same syndromes, while words in different cosets have different syndromes, we can identify a coset by its syndrome; the syndrome of a coset is the syndrome of any word in the coset. Thus if the code has length n and dimension k then the 2n–k words of length n – k each occurs as the syndrome of exactly one of the 2n–k cosets.

Example 2.11.5 The code C of Example 2.11.1 has length n = 4 and dimension k = 2. The cosets of C (listed in Example 2.11.1) contain all 2n = 24 = 16 words of length n = 4. There are 2n–k = 24–2 = 22 = 4 words of length n – k = 2; each one is the syndrome of exactly one of the 2n–k = 4 cosets of C.
< previous page | page_58 | next page >


page_59

< previous page | page_59 | next page >

Page 59

To calculate the syndrome of a particular coset, we can choose any word w in that coset. Then wH will be the syndrome of that coset. For MLD, we want a word of least weight in the coset to use as the error pattern. In the examples in the last section, we carefully arranged the cosets so that a word of least weight was on top, or listed first. Any word of least weight in a coset is called a coset leader. If there was more than one candidate for coset leader, we selected one arbitrarily when doing CMLD.

Example 2.11.6 Again let C be the code of Example 2.11.1. For each coset, we calculate the syndrome, using the coset leader, and display the results in the following table.

Coset leader u | Syndrome uH
0000 | 00
1000 | 11
0100 | 01
0010 | 10

Note again that each word of length 2 occurs once and only once as a syndrome.

The table in the Example 2.11.6, which matches each syndrome with its coset leader, is called a standard decoding array, or SDA. To construct an SDA, first list all the cosets for the code, and choose from each coset a word of least weight as coset leader u. Then find a parity check matrix for the code and, for each coset leader u, calculate its syndrome uH. A quicker way to construct an SDA, given the parity check matrix H and distance d for the code C would be to generate all error patterns e with wt(e) ≤ [image] (d – 1)/2 [image] and compute the syndrome s = eH for each one.

Example 2.11.7 We construct an SDA for the code C of Example 2.10.5 (where the cosets of C have already been listed). For each of the first seven cosets we had no choice for coset leader—the top word is the only word of least weight in its coset. But in the last coset, the smallest weight of a word is 2, and that coset contains three words of weight 2, 000101, 001010, and 110000. Using CMLD we could arbitrarily select 000101 as our presumed error pattern. Using IMLD, we would ask for retransmission and place a [image] in that entry of the SDA to so indicate. We can construct the following parity-check matrix for C:

Then we can obtain the following SDA for C assuming CMLD is being used:
< previous page | page_59 | next page >


page_6

< previous page | page_6 | next page >

Page 6

1.3.5 Add a parity check digit to the codewords in the code in Exercise 1.3.4, and use the resulting code C to answer the following questions.

(a) If 1101 is received can we detect an error?

(b) If 1101 is received what codewords were most likely to have been transmitted?

(c) Is any word of length 4 that is not in the code, closest to a unique codeword?

1.3.6 Repeat each codeword in the code C defined in Exercise 1.3.4 three times to form a repetition code of length 9. Find the closest codewords to the following received words:

(a) 001000001

(b) 011001011

(c) 101000101

(d) 100000010

1.3.7 Find the maximum number of codewords of length n = 4 in a code in which any single error can be detected.

1.3.8 Repeat Exercise 1.3.7 for n = 5, n = 6, and for general n.

1.4 Information rate

After the last section it is apparent that the addition of digits to codewords may improve the error correction and detection capabilities of the code. However, clearly the longer the codewords, the longer it takes to transmit each message. The information rate (or just rate) of a code is a number that is designed to measure the proportion of each codeword that is carrying the message. The information rate of a code C of length n is defined to be (for binary codes)

Since we may assume that 1 ≤ |C| ≤ 2n, it is clear that the information rate ranges between 0 and 1; it is 1 if every word is a codeword and 0 if |C| = 1.

For example, the information rates of the codes C 1, C 2 and C 3 in the previous section are 1, 1/3 and 2/3 respectively. Each of these information rates seems sensibly related to their respective codes, since the first 2 digits of the 6 in each codeword in C 2 can be considered to carry the message, as can the first 2 digits of the 3 in each codeword in C 3.

Exercises

1.4.1 Find the information rate for each of the codes in Exercises 1.3.4, 1.3.5 and 1.3.6.
< previous page | page_6 | next page >


page_60

< previous page | page_60 | next page >

Page 60

Error pattern | Syndrome uH
000000 | 000
100000 | 110
010000 | 011
001000 | 111
000100 | 100
000010 | 010
000001 | 001
000101 | 101

Note that the syndromes are precisely all words in K 3. The coset C always has the zero word as its coset leader and always has syndrome 0. The chosen coset leader for the last coset, u = 000101, gives us syndrome uH = 101, which is the sum of rows 4 and 6 of H, the position with 1's in the error pattern u. Using IMLD, this entry would instead be [image].

Exercises

2.11.8 Construct an SDA assuming IMLD for each of the codes in Exercise 2.10.6.

2.11.9 Construct an SDA assuming IMLD for each of the codes in Exercise 2.10.7.

2.11.10 Construct an SDA assuming IMLD for each of the codes in Exercise 2.10.8.

2.11.11 Prove Theorem 2.11.4.

Finally we can do some decoding. Once we suffer the tedious construction of an SDA, it is easy to use MLD. When we receive a word w, we first calculate the syndrome wH. Then we find the coset leader u next to the syndrome wH = uH in the SDA. We conclude that v = w + u was the most likely codeword sent.

Example 2.11.12 Let C be the code of Example 2.11.1. An SDA appears in Example 2.11.6. The parity-check matrix H is in Example 2.11.3. Assume that w = 1101 is received. Then the syndrome is wH = 11, directing us to the second row of the SDA, where the coset leader is u = 1000. We conclude that v = w + u = 0101 was sent. If w = 1111 is received, then wH = 01 = uH for u = 0100 from the SDA. We decode w as v = w + u = 1011. These results are the same as found in Example 2.11.1.

For w = 1101 received, we decoded v = 0101 as the word sent. The calculations

d(0000, 1101) = 3 | d(0101, 1101) = 1
d(1011, 1101) = 2 | d(1110, 1101) = 2

give the distances between w and each codeword in C, and show that indeed v = 0101 is the closest word in C to w.
< previous page | page_60 | next page >


page_61

< previous page | page_61 | next page >

Page 61

For w = 1111 received, however, the same calculations

d(0000, 1111) = 4 | d(0101, 1111) = 2
d(1011, 1111) = 1 | d(1110, 1111) = 1

reveal a tie for the closest word in C to w. This is not surprising, since there was a choice for a coset leader in the coset containing w. We are doing CMLD, so we arbitrarily choose a coset leader, which in effect arbitrarily selected one word in C closest to w.

Example 2.11.13 Let C be the code of Example 2.10.5. An SDA was constructed in Example 2.11.7. We do some decoding using this SDA. Suppose we receive w = 110111. Then wH = 010, which directs us to the sixth row of the SDA. The coset leader in that row is u = 000010. Thus CMLD concludes that v = w + u = 110111 + 000010 = 110101 was the codeword sent. Now suppose w = 110000 is received. The syndrome wH = 101 directs us to the last row of the SDA where the coset leader is u = 000101. We decode w as v = w + u = 110000 + 000101 = 110101. Had we chosen the word u′ = 001010 as the coset leader for the last coset, then we would instead decode w as w + u′ = 110000 + 001010 = 111010.

Exercises

2.11.14 Continuing the last example with w = 110000 received. Decode assuming that u″ = 110000 had been chosen as the coset leader for the last coset.

2.11.15 Refer to Example 2.11.13 with w = 110111 received. Check that in fact v = 110101 is the closest codeword in C to w.

2.11.16 Again refer to Example 2.11.13 with w = 110000 received. Find all the codewords in C closest to w.

2.11.17 Repeat the decoding in Exercise 2.11.2 using the SDA in Example 2.11.7.

2.11.18 For the code in Example 2.11.13 above, decode the following received words w.

(a) 011101

(b) 110101

(c) 111111

(d) 000000

2.11.19 For each of the following codes, use the SDA to decode the given received words. (The SDA's for these codes were constructed in Exercises 2.11.8 and 2.11.9.)

(a) C = {0000, 1001, 0101, 1100}

(i) w = 1110

(ii) w = 1001

(iii) w = 0101

(b) C = {00000, 10100, 01011, 11111}

(i) w = 10101

(ii) w = 01110

(iii) w = 10001

(c) C = <111000, 001110, 100011>

(i) w = 101010

(ii) w = 011110

(iii) w = 011
< previous page | page_61 | next page >


page_62

< previous page | page_62 | next page >

Page 62

2.11.20 Let C be the code with the parity-check matrix

Decode (a) 110100, (b) 111111, (c) 101010, and (d) 000110.

2.11.21 Let C be the code of length 7 which has as a parity-check matrix the 7 × 3 matrix H whose rows are all nonzero words of length 3.

(a) Construct an SDA for C.

(b) Decode 1010101.

If we want to construct an SDA when using IMLD, we can proceed as follows. If a word w is received, then the number of words in the code C closest to w is the same as the number of error patterns in the coset C + w of least weight. If in some coset of C there is more than one word of smallest weight, then this coset and its syndrome are omitted from the SDA when using IMLD. Furthermore, the weight of a coset leader is the number of errors corrected by MLD when a word in that coset is received. If this weight is excessively high, then we may decide to eliminate this coset and its syndrome from the SDA in IMLD even though there is only one word of least weight in that coset. To use the shortened SDA for IMLD, if the received word has a syndrome which does not occur in the SDA, then we request a retransmission.

In practice, it may not be unusual to have on the order of 250, about 1.126 × 1015 coset leaders and syndromes, which makes the SDA for an arbitrary linear code an unmanageable list. Thus, in practice, we have not solved the problem of decoding using MLD. As we will see later, however, MLD is computationally effective if the linear code is constructed to certain specifications. Indeed, one goal of coding theory is to construct codes which are easy to decode using MLD.

2.12 Reliability of IMLD for linear codes

Let C be a linear code of length n and dimension k. Recall that θp(C, v) is the probability that if v in C is sent over a BSC of probability p, then IMLD will correctly conclude that v was sent.

For each unique coset leader u and for each codeword v in C, v + u is closer to v than to any other codeword. Also, if w ≠ v + u for some codeword v and some unique coset leader u then w is at least as close to some other codeword as it is to v. So for a linear code, the set L(v) of words that are closer to v than to any other

< previous page | page_62 | next page >


page_63

< previous page | page_63 | next page >

Page 63

codeword is

L(v) = {w | w = v + u where u is a unique coset leader}.

If w = v + u then θp(v, w) depends only on wt(u); therefore, for a linear code C, θp(C, v) does not depend on v. We denote this common value by θp(C), and so

Thus, to find the reliability of a linear code, we need be concerned only with the unique coset leaders. Simply calculate the probability of each unique coset leader occurring as an error pattern, then sum these probabilities to get θp(C).

Notice that we have also shown that for a linear code, the set of error patterns that can be corrected using IMLD is equal to the set of unique coset leaders.

Example 2.12.1 Let C be the code of Example 2.10.5. Using IMLD there is one coset leader of weight 0 and six of weight 1. Thus

θp(C) = p 6 + 6p 5(1 – p).

Exercises

2.12.2 Calculate θp(C) for each of the codes in Exercises 2.10.6, 2.10.7, 2.10.8.
< previous page | page_63 | next page >


page_64

< previous page | page_64 | next page >
Page 64 This page intentionally left blank.
< previous page | page_64 | next page >


page_65

< previous page | page_65 | next page >

Page 65

Chapter 3
Perfect and Related Codes

3.1 Some bounds for codes

We now turn our attention to the problem of determining how many words a linear code of given length n and distance d can have. This problem is far from solved in general, though it has been settled for certain values of n and d. We can, however, find some bounds on the size of a code with these given parameters.

Recall that if t and n are integers, 0 ≤ t ≤ n, then the symbol

is just the number of ways that an unordered collection of t objects can be chosen from a set of n objects. Thus [image] is the number of words of length n and weight t.

Theorem 3.1.1 If 0 ≤ t ≤ n and if v is a word of length n, then the number of words of length n of distance at most t from v is precisely

Since there are 2n words of length n, setting t = n in Theorem 3.1.1 yields

Exercises

3.1.2 Illustrate Theorem 3.1.1 for v = 10110 and t = 3 by listing all words in K 5 of distance at most 3 from v, and then check that Theorem 3.1.1 does give the correct number of such words.

To find all words of a given distance t from a fixed word v, we simply add to v, all words of weight t. There are [image] such words. If C is a code of length

< previous page | page_65 | next page >


page_66

< previous page | page_66 | next page >

Page 66

n and distance d = 2t + 1, then there is no word w at distance at most t from two different codewords v 1 and v 2. Indeed, if d(w, v 1) ≤ t and d(w, v 2) ≤ t with v 1 ≠ v 2, then

d(v 1, v 2) ≤ d(v 1, w) + d(w, v 2) ≤ 2t < d = 2t + 1,

which is impossible since C has minimum distance d. Thus, if C has length n and distance 2t + 1, the list of words in Kn at a distance at most t from a codeword v 1 has no codewords in common with the list of words a distance at most t from a codeword v 2, v 1 ≠ v 2. This establishes the following result.

Theorem 3.1.3 (Hamming bound) If C is a code of length n and distance d = 2t + 1 or 2t + 2 then

or

The Hamming bound is an upper bound for the number of words in a code (linear or not) of length n and distance d = 2t + 1. Note that t = [image] (d – 1)/2 [image], so, by Theorem 1.12.9, such a code will correct all error patterns of weight less than or equal to t.

Example 3.1.4 We compute an upper bound for the size or dimension k of a linear code C of length n = 6 and distance d = 3. From d = 3 = 2t + 1 we get t = 1. The Hamming bound gives

But |C| must be a power of 2, so |C| ≤ 8, and thus k ≤ 3.

Exercises

3.1.5 Find an upper bound for the size or dimension of a linear code with the given values of n and d.

(a) n = 8, d = 3

(b) n = 7, d = 3

(c) n = 10, d = 5

(d) n = 15, d = 3

(e) n = 15, d = 5

(f) n = 23, d = 7

3.1.6 Verify the Hamming bound for the linear code C with the given generator matrix.
< previous page | page_66 | next page >


page_67

< previous page | page_67 | next page >

Page 67

Recall that from Section 2.7 and Theorem 2.9.1 we know that the parity check matrix H of an (n, k, d) linear code is an n by n – k matrix such that every d – 1 rows of H are independent. Since the rows have length n – k, you can never have more than n – k independent row vectors. Hence d – 1 ≤ n – k or equivalently k ≤ n – d + 1. This establishes the following result, which is known as the Singleton bound.

Theorem 3.1.7 (Singleton bound) For any (n, k, d) linear code, d – 1 ≤ n – k.

The Singleton bound in one sense is much weaker than the Hamming bound. For example, if n = 15 and d = 5 then Theorem 3.1.7 implies that k ≤ 11, whereas Theorem 3.1.3 (Hamming bound) implies that k ≤ 8. However some codes do attain equality in the Singleton bound, so the Singleton bound is used to define an important and useful class of codes called maximum distance separable codes.

A linear (n, k, d) code is said to be a maximum distance separable (or MDS) code if d = n – k + 1 (or k = n – d + 1). There are several equivalent characterizations of MDS codes.

Theorem 3.1.8 For a (n, k, d) linear code C, the following are equivalent:

(1) d = n – k + 1,

(2) every n – k rows of the parity check matrix are linearly independent,

(3) every k columns of the generator matrix are linearly independent, and

(4) C is MDS.

Proof: Theorem 3.1.7 states that d ≤ n – k + 1; but d ≥ n – k + 1 iff every n – k rows of the parity check matrix are independent. Thus (1) and (2) are equivalent. For (3), note that if d = n – k + 1, no nonzero codeword can have more than k – 1 zeros in it. However, k columns of the k × n generator matrix are linearly dependent iff some nonzero codeword has k zeros in those coordinate positions. The last statement is relatively easy to see and is left to the exercises.

Corollary 3.1.9 The dual of an (n, k, n – k + 1) MDS code is an (n, n – k, k + 1) MDS code.

We will encounter MDS codes later when we study Reed-Solomon codes.

Exercises

3.1.10 Columns 2, 3, and 5 of the generator matrix

are linearly dependent. Find a codeword which has zeros in positions 2, 3, and 5
< previous page | page_67 | next page >


page_68

< previous page | page_68 | next page >

Page 68

3.1.11 Show that if a k × n generator matrix has k linearly dependent columns then there is a nonzero codeword with zeros in those k positions.

We still would like to construct codes for given parameters n, k and d. Upper bounds rule out some parameter values for example the Hamming bound says that a code of length n = 15 and distance d = 5 can not have dimension k = 10. However, this bound does not rule out the possibility of a (15, 8, 5) code existing.

How would we go about finding a (15, 8, 5) code? In general this is a very difficult problem. One approach is to find the parity check matrix for such a code. That is, assuming r = n – k, we must find n vectors of length r to form the rows of H such that every set of d – 1 of these vectors is linearly independent.

Example 3.1.12 Let n = 15, k = 6, and d = 5. Then r = 15 – 6 = 9. So we wish to find 15 nonzero vectors of length 9 with the property that any 4 of these are linearly independent. Finding the first 9 rows is easy: take the 9 × 9 identity matrix I 9.

Suppose we have some how found 3 more vectors for a total of 12 rows and so we have,

Before searching for the next vector we notice that the following counting argument tells us that one must exist. Among all 29 vectors, we cannot select the zero vector nor any of the 12 choosen so far. This rules out 1 + 12 vectors. We also rule out any vector which can be written as the sum of 2 or 3 of these vectors, as this would create a dependent set of 3 or 4 vectors respectively. This rules out at most [image] additional vectors. However, any remaining vector can be selected.

Since

we know that we can find yet another vector. For example one could choose the vector 010101010 to be the next row of H. The chore of finding the remaining rows of H is left to Exercise 3.1.21.

Example 3.1.12 (and the related exercises) show that a (15, 6, 5) code exists. This establishes a lower bound on the maximum size (or dimension) of a linear code with n = 15 and d = 5, i.e., 6 ≤ k ≤ 8.

The next result formalizes the approach of Example 3.1.12 to constructing linear codes (and thus establishing lower bounds). The proofs are left to Exercise 3.1.22.
< previous page | page_68 | next page >


page_69

< previous page | page_69 | next page >

Page 69

Theorem 3.1.13 (Gilbert-Varshamov bound) There exists a linear code of length n, dimension k, and distance d if

Corollary 3.1.14 If n ≠ 1 and d ≠ 1, then there exists a linear code C of length n and distance at least d with

Example 3.1.15 Does there exist a linear code of length n = 9, dimension k = 2, and distance d = 5?

To determine if such a code exists, we use the Gilbert-Varshamov bound:

and 2n–k = 29–2 = 27 = 128. Since 93 < 128, such a code exists.

Example 3.1.16 What is a lower and an upper bound on the size or the dimension, k, of a linear code with n = 9 and d = 5?

To find a lower bound for the most number of codewords such a code C could have, we use Corollary 3.1.14:

Since |C| is linear, it is a power of 2, and thus |C| ≥ 4.

To find an upper bound for |C|, we use the Hamming bound:

Since |C| is linear, it is a power of 2, and thus |C| ≤ 8.

Combining the bounds, a linear code with parameters (9, k, 5) with 4 codewords exists, but no (9, k, 5) linear code with more than 8 codewords exists.

Example 3.1.17 Does there exist a (15, 7, 5) linear code? Again we can try to use the Gilbert-Varshamov bound to answer this question.
< previous page | page_69 | next page >


page_7

< previous page | page_7 | next page >

Page 7

1.5 The effects of error correction and detection

To exemplify the dramatic effect that the addition of a parity-check digit to a code can have in recognizing when error occur, we consider the following codes.

Suppose that all 211 words of length 11 are codewords; then no error is detected. Let the reliability of the channel be p = 1 – 10−8 and suppose that digits are transmitted at the rate of 107 digits per second. Then the probability that a word is transmitted incorrectly is approximately 11 p 10(1 − p), which is about 11/108. So about

are transmitted incorrectly without being detected. That is one wrong word every 10 seconds, 6 a minute, 360 an hour, or 8640 a day! Not too good.

Now suppose that a parity-check digit is added to each codeword, so the number of 1's in each of the 2048 codewords is even. Then any single error is always detected, so at least 2 errors must occur if a word is to be transmitted incorrectly without our knowledge. The probability of at least 2 errors occurring is 1 − p 12 − 12p 11(1 − p) which can be approximated by [image] which for p = 1 − 10−8 is about [image]. Now approximately [image] words per second are transmitted incorrectly without being detected. That is about one error every 2000 days!

So if we are willing to reduce the information rate by lengthening the code from 11 to 12 we are very likely to know when errors occur. To decide where these errors have actually occurred, we may need to request the retransmission of the message. Physically this means that either transmission must be held up until confirmation is received or messages must be stored temporarily until retransmission is requested; both alternatives may be very costly in time or in storage space. It may also be that retransmission is impractical, such as with the Voyager mission and when using compact discs. Therefore, at the expense of further increase in wordlength, it may well be worth incorporating error-correction capabilities into the code. Introducing such capabilities may also make encoding and decoding more difficult, but will help to avoid the hidden costs in time or space mentioned above.

One simple scheme to introduce error-correction is to form a repetition code where each codeword is transmitted three times in succession. Then if at most one error is made per 33 digit codeword, at least two of the three transmissions will be correct. Since the comparisons of the three 11 digit words is relatively simple, the only real trade-off for being able to correct one error is an information rate of 1/3 instead of 1.

Still 1/3 is only 1/3. Perhaps we could do better. We will see later that it is possible to add only 4 extra digits to each 11 digit codeword and still be able to correct any single error. This produces a code with information rate 11/15, a

< previous page | page_7 | next page >


page_70

< previous page | page_70 | next page >

Page 70

and 2n–k = 215–7 = 256. In this case the inequality is not satisfied, so the Gilbert-Varshamov bound does not tell use whether or not such a code exists. In fact, as we shall see later, these are the parameters of the 2 error-correcting BCH code, so such a code does exist.

Exercises

3.1.18 For each part of Exercise 3.1.5, let k = 2d and decide, if possible, whether or not a linear code with the given parameters exists. Find a lower and upper bound for the maximum number of codewords such a code can have, assuming that k is unrestricted.

3.1.19 Find a lower and an upper bound for the maximum number of codewords in a linear code of length n and distance d where

(a) n = 15, d = 5

(b) n = 15, d = 3

(c) n = 11, d = 3

(d) n = 12, d = 3

(e) n = 12, d = 4

(f) n = 12, d = 5.

3.1.20 Is it possible to have a linear code with parameters (8, 3, 5)?

3.1.21 Find a (15, 6, 5) code by constructing the parity check matrix. (See Example 3.1.12, each of the 3 missing vectors must have weight at least 4. Why?)

3.1.22 Let Hi be any i × (n – k) matrix with no d – 1 rows linearly dependent.

(a) Prove that there are at most

words in Kn–k which are linear combinations of at most d – 2 rows of Hi.

(b) Prove that if Ni < 2n–k, then a row can be added in such a way that no d – 1 rows of the resulting matrix are linearly dependent.

(c) Prove the Gilbert-Varshamov bound.

(d) Prove Corollary 3.1.14.

3.2 Perfect codes

A code C of length n and odd distance d = 2t + 1 is called a perfect code if C attains the Hamming bound of Theorem 3.1.3; that is, if

Unfortunately, there are not many linear perfect codes, but the ones that do exist are quite useful. The main problem in finding linear perfect codes is that the number [image] must be a power of 2 (since |C| is a power of 2).
< previous page | page_70 | next page >


page_71

< previous page | page_71 | next page >

Page 71

Example 3.2.1 Let t = 0. Then [image], so [image]. The only code with 2n codewords of length n is C = Kn. Kn is a perfect code.

Example 3.2.2 Let n = 2t + 1. Then

Thus

and, from n = 2t + 1,

Therefore

Hence

Thus any perfect code of length and distance 2t + 1 has exactly 2 codewords. Among linear codes there is only one such code, the repetition code consisting of the zero word and the word in which each digit is 1, and indeed this code is perfect.

The codes in Examples 3.2.1 and 3.2.2, while perfect, are not very interesting. They are called the trivial perfect codes.

Example 3.2.3 Let n = 7 and d = 3. Then t = 1 and

Thus, there may exist a linear perfect code with n = 7 and d = 3. In the next section we shall see that there is such a code, the Hamming code.

Example 3.2.4 Let n = 23 and d = 7. Then t = 3, and

This shows that a linear perfect code with n = 23 and d = 7 may exist. In a later section, we shall see that such a code does exist, namely the Golay code.
< previous page | page_71 | next page >


page_72

< previous page | page_72 | next page >

Page 72

Exercises

3.2.5 Show that for n = 2r – 1, [image].

3.2.6 Can there exist perfect codes for the following values of n and d?

(a) n = 15, d = 3

(b) n = 31, d = 3

(c) n = 15, d = 5

The possible lengths and distances for a perfect code were determined by Tietäväiren and van Lint in 1973. The proof of their result is beyond the scope of these notes.

Theorem 3.2.7 If C is a non-trivial perfect code of length n and distance d = 2t + 1, then either n = 23 and d = 7, or n = 2r – 1 for some r ≥ 2 and d = 3.

If a linear code of length n has distance d = 2t + 1, then, by Theorem 1.12.9, C will correct all error patterns of weight less than or equal to t = (d – 1)/2. Thus every word of length n and weight less than or equal to t is a coset leader. There are precisely [image] such words. But this is precisely the number of cosets if the code is perfect. We have proved another theorem.

Theorem 3.2.8 If C is a perfect code of length n and distance d = 2t + 1, then C will correct all error patterns of weight less than or equal to t, and no other error patterns.

We can interpret Theorem 3.2.8 as saying that each of the 2n words in Kn lies within distance t of exactly one codeword. This property enables us, for example, to count the number of codewords of minimum non-zero weight in a perfect code.

A perfect code which corrects all error patterns of weight less than or equal to t is called a perfect t-error correcting code. From Theorem 3.2.7 the only possible values for t here are t = 1 and t = 3. We examine the case t = 1 in the next section.

3.3 Hamming codes

Finally it's time to design a code. We consider an important family of codes which are easy to encode and decode, and which correct all single errors.

A code of length n = 2r – 1, r ≥ 2, having parity check matrix H whose rows consist of all nonzero vectors of length r is called a Hamming code of length 2r – 1.

Example 3.3.1 One possiblity for a parity check matrix for a Hamming code of length 7 (r = 3) is
< previous page | page_72 | next page >


page_73

< previous page | page_73 | next page >

Page 73

From Algorithm 2.5.7, a generator matrix for Hamming code of length 7 is, therefore,

Thus the code has dimension 4 and contains 24 = 16 codewords. Theorem 2.9.1 can be used to find the distance of the code, which is 3. The information rate is 4/7. In Exercise 2.6.12, we encoded some messages using this code. There are other possibilities for a parity check matrix for a Hamming code of length 7, but all yield equivalent codes.

Since the parity check matrix H for a Hamming code C contains all r rows of weight one, the r columns of H are linearly independent. Thus a Hamming code has dimension 2r – 1 – r and contains [image] codewords.

No row of H is the zero word, so no single row of H is linearly dependent. Thus C has distance at least 2. No two rows of H are equal, so no two rows of H are linearly dependent. Thus C has distance at least 3. But H contains the rows 100...0, 010...0, and 110...0, which form a linearly dependent set. Therefore, by Theorem 2.9.1, a Hamming code has distance d = 3.

Now for n = 2r – 1 and d = 2t + 1 = 3 (so t = 1),

so Hamming codes are perfect codes. By Theorem 3.2.8, Hamming codes are perfect single error-correcting codes.

It is trivial to construct an SDA for a Hamming code. All single errors are corrected so all words of length 2r – 1 and weight one are error patterns that are corrected, and hence must must be coset leaders. Since if e is an error pattern then eH sums the rows of the parity check matrix H corresponding to positions where errors occurred, and since H has 2r – 1 rows, we have the following as an SDA for a Hamming code:

coset leader | syndrome
000...0 | 00...0
 | H

Example 3.3.2 For the Hamming code in Example 3.3.1, decode w = 1101001. The syndrome is wH = 011, which is the fourth row of H. Thus the coset leader u is the fourth row of I 7: u = 0001000. We decode w as w + u = 1100001.
< previous page | page_73 | next page >


page_74

< previous page | page_74 | next page >

Page 74

Exercises

3.3.3 Find a generator matrix in standard form for a Hamming code of length 15, then encode the message 11111100000.

3.3.4 Construct an SDA for a Hamming code of length 7, and use it to decode the following words:

(a) 1101011

(b) 1111111

(c) 0011010

(d) 0101011

(e) 0100011

(f) 0001011

3.3.5 Construct an SDA for a Hamming code of length 15, and use it to decode the following words:

(a) 01010 01010 01000

(b) 11110 00101 10110

(c) 11100 01110 00111

(d) 11100 10110 00000

(e) 00011 10100 00110

(f) 11001 11001 11000

3.3.6 Show that each of the following is a parity check matrix for a Hamming code of length 7, and that the codes are both equivalent to the one in Example 3.3.1.

3.3.7 Prove that all Hamming codes of a given length are equivalent.

3.3.8 Is the following matrix the transpose of a parity check matrix for a Hamming code of length 15?

3.3.9 Show that the Hamming code of length 2r – 1 for r = 2 is a trivial code.

3.3.10 Use the Hamming code of length 7 in Example 3.3.1 and the message assignment in Exercise 2.6.12. Decode the following message received:

1010111, 0110111, 1000010, 0010101, 1001011, 0010000, 1111100.
< previous page | page_74 | next page >


page_75

< previous page | page_75 | next page >

Page 75

3.4 Extended codes

Sometimes increasing the length of a code by one digit, or perhaps a few digits, result in a new code with improved error detection or error correction capabilities which are worth the price of a lower information rate. We consider one simple possibility in this section.

Let C be a linear code of length n. The code C* of length n + 1 obtained from C by adding one extra digit to each codeword in order to make each word in the new code have even weight is called the extended code of C.

In Example 1.3.3 we constructed the extended code of K 2, and the reader did the same for K 3 in Exercise 1.3.5.

If the original code C has a k × n generator matrix G, then the extended code C* has k × (n + 1) generator matrix.

G* = [G, b],

where the last column b of G* is appended so that each row of G* has even weight.

A parity check matrix for C* can be constructed from G* using Algorithm 2.5.7. But there is an easier way if we are given a parity check matrix H for the original code C. In this case, the extended code C* has a parity check matrix

where j is the n × 1 column of all ones. Note that H* is an (n + 1) × (n + 1 – k) matrix. Since H has rank n – k, the last row of H* ensures that H* has rank n – k + 1. Moreover,

Now GH = 0 and Gj sums the ones in each row of G. From the definition of b, it follows that Gj + b = 0. Therefore G* H* = 0. By Theorem 2.7.6, G* and H* are indeed generating and parity check matrices respectively for the linear code C*.

Example 3.4.1 Let C be the linear code with generator matrix

Then
< previous page | page_75 | next page >


page_76

< previous page | page_76 | next page >

Page 76

is a parity check matrix for C by Algorithm 2.5.7. So we obtain the following generating and parity check matrices for the extended code:

If v is a word in the original code C and if v* is the corresponding word in the extended code C* then

Therefore if the distance d of C is odd then the distance of C* is d + 1, but if d is even then the distance of C* is d. So an extended code is of use only when d is odd, in which case it corrects no more errors than C but will detect one more error. Notice then that there is no point in extending a code twice.

Example 3.4.2 Assume C has distance d = 5. Then C* has distance d* = 6. By Theorem 1.11.14, C detects all nonzero error patterns of weight less than or equal to d – 1 = 4, and C* detects all nonzero error patterns of weight less than or equal to d* – 1 = 5. By Theorem 1.12.9, C corrects all error patterns of weight less than or equal to [image] (d – 1)/2 [image] = [image] 4/2 [image] = 2, and C* corrects all error patterns of weight less than to equal to [image] (d* – 1)/2 [image] = [image] 5/2 [image] = 2.

Exercises

3.4.3 Find generating and parity check matrices for an extended Hamming code of length 8.

3.4.4 Construct an SDA for an extended Hamming code of length 8, and use it to decode the following words.

(a) 10101010

(b) 11010110

(c) 11111111

3.4.5 Show that an extended Hamming code of length 8 is a self-dual code (that is, show C = C⊥).

3.4.6 Find a formula for the distance d* of the extended code C* in terms of the distance of the original code C.

3.4.7 Let C be a Hamming code of length 15. Find the number of error patterns that Theorem 1.11.14 guarantees the extended code C* will detect, and the number of error patterns Theorem 1.12.9 guarantees C* will correct. How may error patterns does C* correct?
< previous page | page_76 | next page >


page_77

< previous page | page_77 | next page >

Page 77

3.5 The extended Golay code

In this and the next two sections we construct and decode two codes which will correct three or fewer errors. The extended Golay code, discussed in this and the next section, was in fact used in the Voyager spacecraft program which, in the early 1980's, brought us those marvelous close-up photographs of Jupiter and Saturn.

Let B be the 12 × 12 matrix

Let G be the 12 × 24 matrix G = [I, B], where I is the 12 × 12 identity matrix. The linear code C with generator matrix G is called the extended Golay code and will be denoted by C 24.

As an aid to remembering B, note that the 11 × 11 matrix B 1 obtained from B by deleting the last row and column has a cyclic structure. The first row of B 1 is 11011100010. The second row is obtained from the first by shifting each digit one position to the left and moving the first digit to the end. The third row is obtained from the second row in the same way, and so on for the remaining rows. Thus B may be remembered as the matrix

where j is the word of all ones of length 11. By inspection, we see that BT = B; that is, B is a symmetric matrix.

We now list seven important facts about the extended Golay code C 24 with generator matrix G = [I, B]:

(1) C 24 has length n = 24, dimension k = 12 and 212 = 4096 codewords. This is clear upon inspection of G.

(2) A parity check matrix for C 24 is the 24 × 12 matrix
< previous page | page_77 | next page >


page_78

< previous page | page_78 | next page >

Page 78

Algorithm 2.5.7 yields this fact.

(3) Another parity check matrix for C 24 is the 24 × 12 matrix

To see this, note first that each row of B has odd weight (7 or 11). The scalar (dot) product of any row with itself is therefore 1. Next, a manual check shows that the scalar product of the first row of B with any other row is 0. From the cyclic structure of B 1 it follows that the scalar product of any two different rows of B is 0. Thus B BT = I. But BT = B, so B 2 = B BT and,

We shall use both parity check matrices to decode C 24.

(4) Another generator matrix for C 24 is the 12 × 24 matrix [B, I].

(5) C 24 is self-dual; that is,

(6) The distance of C is 8.

(7) C 24 is a three-error-correcting code.

The proofs of facts (4) and (5) are requested in exercises. We will give a proof of fact (6) which, in the bargain, contains further useful information about the code C 24. The proof is divided into three stages:

Stage I The weight of any word in C 24 is a multiple of 4. To see this, note first that the rows of G have weight 8 or 12. Let v be a word in C 24 which is the sum v = ri + rj of two different rows of G. The rows of B are orthogonal; hence the rows of G are orthogonal. Therefore ri and rj have an even number, say 2x, of ones in common. Thus

wt(v) = wt(ri) + wt(rj) – 2(2x)

is a multiple of 4.

Now suppose the word v in C 24 is the sum v = ri + rj + rk of three different rows of G. Let v 1 = ri + rj. Since C 24 is self-dual, v 1 and rk have scalar product 0, and hence an even number, say 2y, of ones in common. Thus

wt(v) = wt(v 1) + wt(rk) – 2(2y)

is a multiple of 4. Continuing in this vein (formally, by induction) we see that if v in C 24 is a linear combination of rows of G, then wt(v) must be a multiple of 4.

Stage II The first eleven rows of G are codewords in C 24 of weight 8, so the distance of C 24 must be either 4 or 8.
< previous page | page_78 | next page >


page_79

< previous page | page_79 | next page >

Page 79

Stage III We rule out words of weight 4 being codewords in C 24. Let v be a nonzero codeword in C 24, and suppose wt(v) = 4. Then v = u 1[I, B] and v = u 2[B, I] for some u 1 and u 2 (since both [I, B] and [B, I] generate C 24) and wt(u 1) ≤ 2 or wt(u 2) ≤ 2 (since one half of v must contain at most two 1's). However no sum of one or two row of B has weight at most 3, so wt(v) = wt(ui) + wt(ui B) > 4. Therefore v does not have weight 4.

Exercises

3.5.1 Show that the word of all ones is in C 24. Deduce that C 24 contains no words of weight 20.

3.5.2 Prove fact (4) about C 24.

3.5.3 Prove fact (5) about C 24.

3.5.4 Use Theorem 2.9.1 to verify that C 24 has distance 8.

3.6 Decoding the extended Golay code

We shall now find an algorithm for IMLD for the code C 24. Throughout this section, w denotes the word received, v the closest codeword to w and u the error pattern v + w. For C 24 we want to correct all error patterns of weight at most 3, so we assume that wt(u) ≤ 3. A comma will be placed between the first 12 and the last 12 digits of words in K 24. The error pattern u will be denoted by [u 1, u 2], where u 1 and u 2 each have length 12. Our aim is to determine the coset leader, u of the coset containing w without having to refer to the SDA of C 24

Since we are assuming that wt(u) ≤ 3, either wt(u 1) ≤ 1 or wt(u 2) ≤ 1. Let s 1 be the syndrome of w = v + u using the parity check matrix

Then s 1 = wH = [u 1, u 2]H = u 1 + u 2 B. So if wt(u 2) ≤ 1 then s 1 consists of either a word of weight at most 3 (if wt(u 2) = 0) or a row of B with at most 2 of its digits changed (if wt(u 2) = 1). Similary, if wt(u 1) ≤ 1 then the syndrome

consists of either a word of weight at most 3 or a row of B with at most 2 of its digits changed.

In any case, if u has weight at most 3 then it is easily identified, since at most 3 rows of one of the two parity check matrices can be found to add to the corresponding syndrome. Using this observation we obtain the following decoding

< previous page | page_79 | next page >


page_8

< previous page | page_8 | next page >

Page 8

valuable improvement provided that the extra encoding and decoding costs are not prohibitive.

It is our task, then, to design codes with reasonable information rates, low encoding and decoding costs and some error-correcting or error-detecting capabilities that make the need for retransmission unlikely.

1.6 Finding the most likely codeword transmitted

Suppose that we have an overall view of the transmission process, knowing both the codeword v that is transmitted and the word w that is received. For any given v and w, let φp(v,w) be the probability that if the codeword v is sent over a BSC with reliability p then the word w is received. Since we are assuming that noise is distributed randomly, we can treat the transmission of each digit as an independent event. So if v and w disagree in d positions, then we have n − d digits correctly transmitted and d incorrectly transmitted and thus,

;p(v,w) = pn−d(1 − p)d.

Example 1.6.1 Let C be a code of length 5. Then for any v in C, the probability that v is received correctly is

φp(v,v) = p 5.

Let 10101 be in C. Then

φp(10101, 01101) = p 3(1 − p)2

and if p = .9 then

φ.9(10101, 01101) = (.9)3(.1)2 = .00729.

Exercises

1.6.2 Calculate φ.97(v,w) for each of the following pairs of v and w:

(a) v = 01101101, w = 10001110

(b) v = 1110101, w = 1110101

(c) v = 00101, w = 11010

(d) v = 00000, w = 00000

(e) v = 1011010, w = 0000010

(f) v = 10110, w = 01001

(g) v = 111101, w = 000010.

In practice we know w, the word received but we do not know the actual codeword v that was sent. However each codeword v determines an assignment of probabilities φp(v,w) to words, w. Each such assignment is a mathematical model and we choose the model (that is, the codeword v) which agrees most with

< previous page | page_8 | next page >


page_80

< previous page | page_80 | next page >

Page 80

algorithm. We shall make use of the fact that B 2 = I and

To avoid incorporating both of the parity check matrices into the algorithm, only [image] is used. Of course once u has been determined, w is decoded to the codeword v = w + u. ei is the word of length 12 with a 1 in the ith position and 0s elsewhere, and bi is the ith row of B.

Algorithm 3.6.1 (decoding the extended Golay code)

1. Compute the syndrome s = wH.

2. If wt(s) ≤ 3 then u = [s, 0].

3. If wt(s + b i) ≤ 2 for some row bi of B then u = [s + bi, ei].

4. Compute the second syndrome sB.

5. If wt(sB) ≤ 3 then u = [0, sB].

6. If wt(sB + bi) ≤ 2 for some row bi of B then u = [ei, sB + bi].

7. If u is not yet determined then request retransmission.

The above algorithm requires at most 26 weight calculations in the decoding procedure. (Of course, once u has been determined then no further steps in the algorithm need to be done.)

Example 3.6.2 Decode w = 101111101111, 010010010010. The syndrome is

which has weight 2. Since wt(s) ≤ 3, we find that

u = [s, 0] = 100000000001, 000000000000

and conclude that

v = w + u = 001111101110, 010010010010

was the codeword sent.

Because G = [I, B] is in standard form and any word in K 12 can be encoded as a message (C 24 has dimension 12), the message sent appears in the first 12 digits of the decoded word v. In Example 3.6.2 the message 001111101110 was sent.
< previous page | page_80 | next page >


page_81

< previous page | page_81 | next page >

Page 81

Example 3.6.3 Decode w = 001001001101, 101000101000. The syndrome is

s = wH = 001001001101 + 111000000100 = 110001001001,

which has weight 5. Proceeding to step 3 of the Algorithm 3.6.1 we compute

s + b 1 = 000110001100
s + b 2 = 011111000010
s + b 3 = 101101011110
s + b 4 = 001001100100
s + b 5 = 000000010010.

Since wt(s + b 5) ≤ 2, we find that

u = [s + b 5, e 5] = 000000010010, 000010000000

and conclude that

v = w + u = 001001011111, 101010101000

was the codeword sent.

Example 3.6.4 Decode w = 000111000111, 011011010000. The syndrome is

which has weight 7. Proceeding to step 3, we find wt(s + bi) ≥ 3 for each row bi of B. We continue to step 4; the second syndrome is

sB = 111001111101,

which has weight 9. Forging ahead to step 5 we compute

sB + b 1 = 001110111000
sB + b 2 = 010111110110
sB + b 3 = 100101101010
sB + b 4 = 000001010000.

Since wt(sB + b 4) ≤ 2, we find that

u = [e 4, sB + b 4] = 000100000000, 000001010000

and conclude that

v = w + u = 000011000111, 011010000000

was the codeword sent.
< previous page | page_81 | next page >


page_82

< previous page | page_82 | next page >

Page 82

Exercises

3.6.5 The code is C 24. Find the most likely error pattern if possible, for each of the following received words w.

(a) 111 000 000 000, 011 011 011 011

(b) 111 111 000 000, 100 011 100 111

(c) 111 111 000 000, 101 011 100 111

(d) 111 111 000 000, 111 000 111 000

(e) 111 000 000 000, 110 111 001 101

(f) 110 111 001 101, 111 000 000 000

(g) 000 111 000 111, 101 000 101 101

(h) 110 000 000 000, 101 100 100 000

(i) 110 101 011 101, 111 000 000 000.

3.6.6 Find the most likely error pattern for any word with the given syndromes.

(a) s 1 = 010010000000, s 2 = 011111010000

(b) s 1 = 010010100101, s 2 = 001000110000

(c) s 1 = 111111000101, s 2 = 111100010111

(d) s 1 = 111111111011, s 2 = 010010001110

(e) s 1 = 001101110110, s 2 = 111110101101

(f) s 1 = 010111111001, s 2 = 100010111111

3.6.7 Show that if s or sB has weight 4 then IMLD requires that the word be retransmitted.

3.7 The Golay code

Another interesting three-error-correcting code can be obtained by puncturing C 24, that is, by removing a digit from every word in C 24. The same digit must be removed from each word. We shall remove the last digit.

Let [image] be the 12 × 11 matrix obtained from the matrix B by deleting the last column. Let G be the 12 × 23 matrix G = [I 12, [image]]. The linear code with generator matrix G is called the Golay code and is denoted by C 23. The Golay code has length n = 23, dimension k = 12, and contains 212 = 4096 codewords. Note that the extended code [image] is indeed C 24. C 23 has distance 7. This is most easily seen from the fact that [image] (see Exercise 3.4.6), but can be shown using Theorem 3.2.8 or by modifying the proof that C 24 has distance 8.

The Golay code C 23 is a perfect code (Example 3.2.4) and will correct all error patterns of weight 3 or less, and no others (Theorem 3.2.8). Therefore every received word w is at most distance 3 from exactly one codeword. So if we append the digit 0 or 1 to w forming w0 or w1 respectively so that the resulting word has

< previous page | page_82 | next page >


page_83

< previous page | page_83 | next page >

Page 83

odd weight, then the resulting word is distance at most 3 from a codeword c in C 24 (see Exercise 3.7.10). Decoding to c using Algorithm 3.6.1 and removing the last digit from c then gives the closest codeword to w in C 23.

Algorithm 3.7.1 (decoding the Golay code)

1. Form w0 or w1, whichever has odd weight.

2. Decode wi (i is 0 or 1) using Algorithm 3.6.1 to a codeword c in C 24.

3. Remove the last digit from c.

In practice, the received word w is normally a codeword, however wi formed in step 1 is never a codeword (Why?). If w is a codeword then the syndrome of wi is the last row of H (Why?) so this can easily be checked before implementing Algorithm 3.6.1.

Example 3.7.2 Decode w = 001001001001, 11111110000. Since w has odd weight, form w0 = 001001001001, 111111100000. Then s 1 = 100010111110. Since s 1 = b 6 + e 9 + e 12, w0 is decoded to 001001000000, 111110100000 and so w is decoded to 001001000000, 11111010000.

Exercises

3.7.3 Decode each of the following received words that were encoded using C 23.

(a) 101011100000, 10101011011

(b) 101010000001, 11011100010

(c) 100101011000, 11100010000

(d) 011001001001, 01101101111.

3.7.4 Prove that C 23 has distance d = 7.

3.7.5 Find the reliability of C 23 transmitted over a BSC of probability p.

3.7.6 Determine whether C 23 or C 24 has the greater reliability. Use the same BSC for both.

3.7.7 Use the fact that every word of weight 4 is distance 3 from exactly one codeword (why?) to count the number of codewords of weight 7 in the Golay code. (Hint: for any codeword c, the number of words that have weight 4 and are distance 3 from c is [image].)

3.7.8 Use Exercise 3.7.7 to show that C 24 contains precisely 759 codewords of weight 8. (Hint: each word of weight 5 is at distance at most 3 from exactly one codeword.)

3.7.9 Use Exercises 3.5.1 and 3.7.8 to verify the following weight distribution table for C 24:

weight | 0 | 4 | 8 | 12 | 16 | 20 | 24
number of words | 1 | 0 | 759 | 2576 | 759 | 0 | 1
< previous page | page_83 | next page >


page_84

< previous page | page_84 | next page >

Page 84

3.7.10 Let w be a received word that was encoded using C 23. Append a digit i to w to form a word wi of odd weight. Show that wi is within distance 3 of a codeword in C 24. (Hint: all words in C 24 have even weight.)

3.8 Reed-Muller codes

In this section we briefly consider another important class of codes which includes the extended Hamming code discussed earlier (see also Chapter 9). The rth order Reed-Muller code of length 2m will be denoted by RM(r, m), 0 ≤ r ≤ m. We present a recursive definition of these codes:

(1) RM(0, m) = {00...0, 11...1}, RM(m, m) =

(2) RM(r, m) = {(x, x + y) | x ∈ RM(r, m – 1), y ∈ RM(r – 1, m – 1)}, 0 < r < m.

So RM(m, m) is all words of length 2m and RM(0, m) is just the all ones word (and the zero word).

Example 3.8.1

Rather than use this description of the code, we will give a recursive construction for the generator matrix of RM(r, m), which we will denote by G(r, m). For 0 < r < m, define G(r, m) by

For r = 0 define

G(0, m) = [11...1]

and for r = m, define

Example 3.8.2 The generator matrices for RM(0, 1) and RM(1, 1) are

Example 3.8.3 Let m = 2, then the length is 4 = 22 and for r = 1, 2 we have
< previous page | page_84 | next page >


page_85

< previous page | page_85 | next page >

Page 85

Using Example 3.8.2 we have,

Example 3.8.4 For m = 3, n = 23 = 8, we have

Thus using Example 3.8.3

Exercises

3.8.5 Find the generator matrix G(2, 3).

3.8.6 Find generator matrix G(r, 4), for the codes RM(r, 4) for r = 0, 1, 2.

With this recursive definition it is a simple matter to prove via induction the basic properties of a Reed-Muller code.

Theorem 3.8.7 The r th order Reed-Muller code RM(r, m) defined above has the following properties:

(1) length n = 2m

(2) distance d = 2m-r

(3) dimension

(4) RM(r – 1, m) is contained in RM(r, m), r > 0

(5) dual code RM(m – 1 – r, m), r < m.

Proof: The proofs of these claims all use induction. We leave it as an exercise to show that this theorem holds for all RM(r, m) codes for m = 1, 2, 3, 4. Also, we note that these claims are obviously true for r = 0 and r = m.

First we want to show that RM(r – 1, m) ⊆ RM(r, m). We state with
< previous page | page_85 | next page >


page_86

< previous page | page_86 | next page >

Page 86

Since 1 is the top row of G(1, m – 1) then the all ones vector (1, 1) is the top row vector in (G(1, m – 1), G(1, m – 1)). Thus RM(0, m) = {0, 1} is contained in RM(1, m).

In general since G(r – 1, m – 1) is a submatrix of G(r, m – 1) and G(r – 2, m – 1) is a submatrix of G(r – 1, m – 1) we have obviously

is a submatrix of G(r, m) and thus RM(r – 1, m) is a subcode of RM(r, m).

Next we establish the distance d = 2m–r for RM(r, m), using induction on r.

Since RM(r, m) = {(x, x + y) | x ∈ RM(r, m – 1), y ∈ RM(r – 1, m – 1)} and RM(r – 1, m – 1) ⊆ RM(r, m – 1) then x + y ∈ RM(r, m – 1) and so if x ≠ y, then, by our inductive hypothesis, wt(x + y) ≥ 2m–1–r. Also wt(x) ≥ 2m–1–r. Hence wt(x, x + y) = wt(x + y) + wt(x) ≥ 2 · 2m–1–r = 2m–r. If x = y, then (x, x + y) = (y, 0) but y ∈ RM(r – 1, m – 1) and thus wt(y, 0) = wt(y) ≥ 2m–1–(r–1) = 2m–r.

From the definition of G(r, m), we have

Since [image] and [image] we have,

Finally let

RM(r, m) = {(x, x + y) | x ∈ RM(r, m – 1), y ∈ RM(r – 1, m – 1)}

and let

RM(m – r – 1, m) = {(x′, x′ + y′) | x′ ∈ RM(m– r – 1, m – 1), y′ ∈ RM(m–r–2, m–1)}.

By induction the dual of RM(r, m – 1) is RM(m – r – 2, m – 1) and the dual of RM(r – 1, m – 1) is RM(m – r – 1, m – 1) thus x · y′ = 0, and x′ · y = 0. Also

< previous page | page_86 | next page >


page_87

< previous page | page_87 | next page >

Page 87

since RM(r – 1, m – 1) ⊆ RM(r, m – 1), y · y′ = 0. Hence

We see that every vector in RM(r, m) is orthogonal to every vector in RM(m – r – 1, m). Since

the RM(m – r – 1, m) code is the dual of the RM(r, m) code.

Exercises

3.8.8 Show that Theorem 3.8.7 holds for the codes RM(r, m), 1 ≤ m ≤ 4, constructed in Examples 3.8.1, 3.8.3, 3.8.4 and Exercises 3.8.5, 3.8.6.

We consider the first order Reed-Muller code RM(1, m). Notice that RM(m–2, m) has dimension 2m – m – 1 and has distance 4, length 2m and therefore is an extended Hamming code. By Theorem 3.8.7, RM(1, m) is the dual of this extended Hamming code. We present a decoding algorithm for this code which is quite efficient. We postpone a discussion of a decoding algorithm for general RM(r, m) codes until Chapter 9.

Note that the RM(1, m) code is a small code with a large minimum distance, so a good decoding algorithm is in fact the most elementary: for each received word w, find the codeword in RM(1, m) closest to w. This can be done very efficiently.

Example 3.8.9 Let m = 3, consider the RM(1, 3) code which has length 8 = 23, and 16 = 23 + 1 codewords. The minimum distance is 4. Let

Note that if w is received and d(w, c) < 2 then we decode w to c but if d(w, c) > 6, then d(w, 1 + c) < 2 and we decode w to 1 + c. (Recall 1 is a codeword.) For example, if w = 1000 1111 is received then c = 0000 1111 is the nearest codeword.

< previous page | page_87 | next page >


page_88

< previous page | page_88 | next page >

Page 88

If w = (10101011) is received and we find c = (01010101) with d(w, c) > 6, then c + 1 = 10101010 is the nearest codeword. Thus we have to examine at most half of the codewords in RM(1, m).

In fact, there are very efficient matrix methods to compute these distances which we will present in the next section.

Exercises

3.8.10 Let G(1, 3) be the generator for the RM(1, 3) code, decode the following received words

(a) 0101 1110

(b) 0110 0111

(c) 0001 0100

(d) 1100 1110

3.8.11 Let G(1, 4) be the generator for the RM(1, 4) code, decode the following received words

(a) 1011 0110 0110 1001

(b) 1111 0000 0101 1111

3.9 Fast decoding for RM(1, m)

In this section we present briefly and without justification a very efficient decoding method for RM(1, m) codes. It utilizes the Fast Hadamard Transform to find the nearest codeword. First we need to introduce the Kronecker product of matrices.

Define A × B = [aij B]; that is, entry aij in A is replaced by the matrix aij B.

Example 3.9.1 Let [image]. Then

Now we consider a series of matrices defined as:

for i = 1, 2, ..., m, where H is as in Example 3.9.1.

Example 3.9.2 Let m = 2. Then

(see Example 3.9.1).
< previous page | page_88 | next page >


page_89

< previous page | page_89 | next page >

Page 89

Example 3.9.3 Let m = 3 then

The recursive nature of the construction of RM(1, m) codes suggests that there is a recursive approach to decoding as well. This is the intuitive basis for the following decoding algorithm for RM(1, m).

Algorithm 3.9.4 (decoding the RM(1, m) code) Suppose w is received and G(1, m) is the generator matrix for RM(1, m) code.

1. Replace 0 by – 1 in w forming

2. Compute [image] and [image] for i = 2, 3, ..., m.

3. Find the position j of the largest component (in absolute value) of wm.

Let v(j) ∈ Km be the binary representation of j (low order digits first). Then if the jth component of wm is positive, the presumed message is (1, v(j)), and if it is negative the presumed message is (0, v(j)).
< previous page | page_89 | next page >


page_9

< previous page | page_9 | next page >

Page 9

observation—in this case, which makes the word received most likely. That is, assume v is sent when w is received if

φp(v,w) = max{φp(u,w) : u ∈ C}.

The following theorem provides a criterion for finding such a codeword v.

Theorem 1.6.3 Suppose we have a BSC with 1/2 < p < 1. Let v1 and v2 be codewords and w a word, each of length n. Suppose that v1 and w disagree in d1 positions and v2 and w disagree in d2 positions. Then

φp(v 1,w) ≤ φp(v 2,w) if and only if d 1 ≥ d 2.

Proof: We have already established that φp(v 1,w) ≤ φp(v2,w) iff pn−d 1(1 − p)d 1 ≤ pn−d 2(1 − p)d 2 iff [image] iff d 2 ≤ d 1 [image].

This formally establishes the procedure for correcting words which until now we had adopted as being an intuitively sensible procedure: correct w to a codeword which disagrees with w in as few positions as possible, since such a codeword is the most likely to have been sent, given that w was received.

Example 1.6.4 If w = 00110 is received over a BSC with p = .98, which of the codewords 01101, 01001, 10100, 10101 was the most likely one sent?

v | d (number of disagreements with w)
01101 | 3
01001 | 4
10100 | 2 ← smallest d
10101 | 3

Using the above table, Theorem 1.6.3 says that 10100 was the most likely codeword sent. Note that we don't need to know the precise value of p in order to apply Theorem 1.6.3; we only need to know that p > 1/2.

Exercises

1.6.5 Suppose that w = 0010110 is received over a BSC with reliability p = .90. Which of the following codewords is most likely to have been sent?

1001011, 1111100, 0001110, 0011001, 1101001.

1.6.6 Which of the 8 codewords in the code of Exercise 1.3.6 is most likely to have been sent if w = 101000101 is received?

1.6.7 If C = {01000, 01001, 00011, 11001} and a word w = 10110 is received, which codeword is most likely to have been sent?

1.6.8 Repeat Exercise 1.6.7 after replacing C with {010101, 110110, 101101, 100110, 011001} and w with 101010.
< previous page | page_9 | next page >


page_90

< previous page | page_90 | next page >

Page 90

Example 3.9.5 Let m = 3, and G(1, 3) be the generator matrix for RM(1, 3) (see Example 3.8.9). If w = 10101011 is received, convert w to [image] = (1, –1, 1, –1, 1, –1, 1, 1). Compute:

(see Example 3.9.2 for [image]). The largest component of w 3 is 6 occurring in position 1. Since v(1) = 100 and 6 > 0, then the presumed message is m = (1100).

Suppose w = (10001111). Then [image] and

The largest component of w 3 is –6 occurring in position 4. Since v(4) = 001 and –6 < 0 the presumed message is (0001).

Exercises

3.9.6 Decode the received words in Exercise 3.8.10 using Algorithm 3.9.4 (and Example 3.9.2).

3.9.7 Compute [image] for i = 1, 2, 3, 4.

3.9.8 Decode the received words in Exercise 3.8.11 using Algorithm 3.9.4 (and Exercise 3.9.6).
< previous page | page_90 | next page >


page_91

< previous page | page_91 | next page >

Page 91

Chapter 4
Cyclic Linear Codes

4.1 Polynomials and words

We will find it convenient to represent cyclic codes in terms of polynomials. For this reason we review some needed facts about polynomials (of one variable).

A polynomial of degree n over K is a polynomial a 0 + a 1 x + · · · + a n xn where the coefficients a0, ..., an are elements of K. The set of all polynomials over K is denoted by K[x]. Elements of K[x] will be denoted by f(x), g(x), p(x) and so forth, and the degree of f(x) is denoted by deg(f(x)).

Polynomials over K are added and multiplied in the usual fashion except that since 1 + 1 = 0, we have that x k + x k = 0. This means that the degree of f(x) + g(x) need not be max{deg(f(x)), deg(g(x))}.

Example 4.1.1 Let f(x) = 1 + x + x 3 + x 4, g(x) = x + x 2 + x 3 and h(x) = 1 + x 2 + x 4. Then:

(a) f(x) + g(x) = 1 + x 2 + x 4;

(b) f(x) + h(x) = x + x 2 + x 3;

(c) f(x)g(x) = (x + x 2 + x 3) + x(x + x 2 + x 3) + x 3(x + x 2 + x 3) + x 4(x + x 2 + x 3) = x + x 7.

Exercises

4.1.2 Find the sum and the product of each of the following pairs of polynomials over K:

(a) f(x) = x 5 + x 6 + x 7, h(x) = 1 + x 2 + x 3 + x 4;

(b) f(x) = 1 + x 2 + x 3 + x 8 + x 13, h(x) = 1 + x 3 + x 9,

(c) f(x) = 1 + x, h(x) = 1 + x + x 2 + x 3 + x 4.

4.1.3 Let f(x) = 1 + x. Find:

(a) (f(x))2

(b) (f(x))3

(c) (f(x))4.

4.1.4 Repeat Exercise 4.1.3 for f(x) = 1 + x + x 2.
< previous page | page_91 | next page >


page_92

< previous page | page_92 | next page >

Page 92

4.1.5 List all polynomials over K of degree n, for n = 0, n = 2, n = 3, and n = 4.

4.1.6 Find the number of polynomials over K of degree at most 10.

4.1.7 As you may have noticed in Exercises 4.1.3 (a) and 4.1.4 (a), for any polynomials f(x) and g(x) in K[x],

(f(x) + g(x))2 = (f(x))2 + (g(x))2

since x k + x k = 0. Is there also a special rule in K[x] for

(a) (f(x) + g(x))4,

(b) (f(x) + g(x))3,

(c) (f(x) + g(x))n, for any positive integer n?

The usual long division process works for polynomials over K just as it does for polynomials over the rational numbers.

Algorithm 4.1.8 (Division algorithm) Let f(x) and h(x) be in K[x] with h(x) ≠ 0. Then there exist unique polynomials q(x) and r(x) in K[x] such that

f(x) = q(x)h(x) + r(x),

with r(x) = 0 or deg(r(x)) < deg(h(x)).

The polynomial q(x) is called the quotient, and r(x) is called the remainder. The procedure for finding the quotient and the remainder when h(x) is divided into f(x) is the familiar long division process, but with the arithmetic in K among the coefficients.

Example 4.1.9 Let f(x) = x + x 2 + x 6 + x 7 + x 8 and h(x) = 1 + x + x 2 + x 4. Then

Thus the quotient is q(x) = x 3 + x 4 and the remainder is r(x) = x + x 2 + x 3. We may write f(x) = h(x)(x 3 + x 4) + (x + x 2 + x 3). Note that deg(r(x)) < deg(h(x)) = 4.

Exercises

4.1.10 Find the quotient and remainder when h(x) is divided into f(x) for each of the pairs of polynomials over K in Exercise 4.1.2.
< previous page | page_92 | next page >


page_93

< previous page | page_93 | next page >

Page 93

4.1.11 Find the quotient and remainder in each part when h(x) is divided into f(x).

(a) f(x) = x 2 + x 3 + x 4 + x 8, h(x) = 1 + x 5.

(b) f(x) = 1 + x 10, h(x) = 1 + x 5

(c) f(x) = 1 + x 7, h(x) = 1 + x + x 3

(d) f(x) = 1 + x 15, h(x) = 1 + x 4 + x 6 + x 7 + x 8.

The polynomial f(x) = a 0 + a 1 x + a 2 x 2 + · · · + a n–1 x n–1 of degree at most n–1 over K may be regarded as the word v = a 0 a 1 a 2...a n–1 of length n in Kn. For example if n = 7,

polynomial | word
1 + x + x 2 + x 4 | 1110100
1 + x 4 + x 5 + x 6 | 1000111
1 + x + x 3 | 1101000

Thus a code C of length n can be represented as a set of polynomials over k of degree at most n – 1.

Note that it may be convenient for purposes of representing words by polynomials to number the digits of a word of length n from 0 to n – 1, rather than from 1 to n. The word a 0 a 1 a 2 a 3 of length 4 is represented by the polynomial a 0 + a 1 x + a 2 x 2 + a 3 x 3 of degree 3, for instance.

Example 4.1.12 The code C in the left column of the array is represented by the polynomials in the right column.

codeword c | polynomial c(x)
0000 | 0
1010 | 1 + x 2
0101 | x + x 3
1111 | 1 + x + x 2 + x 3

Exercises

4.1.13 Represent each codeword C in the following codes with polynomials.

(a) C = {000, 001, 010, 011}

(b) C = {00000, 11111}

(c) C = {0000, 0001, 1110}

(d) C = {0000, 1001, 0110, 1111}

(e) C = {00000, 11100, 00111, 11011}

4.1.14 Write out the Hamming code of length 7 generated by the matrix G and then represent this code by polynomials.
< previous page | page_93 | next page >


page_94

< previous page | page_94 | next page >

Page 94

In Exercise 4.1.11(a), the reader computed the remainder r(x) when f(x) = x 2 + x 3 + x 4 + x 8 was divided by h(x) = 1 + x 5. The result was r(x) = x 2 + x 4. By the Division Algorithm, r(x) is unique. Also r(x) has degree less than the degree of the divisor h(x).

We say that f(x) modulo h(x) is r(x) if r(x) is the remainder when f(x) is divided by h(x); we shall write r(x) = f(x) mod h(x). Furthermore, we say that two functions f(x) and p(x) are equivalent modulo h(x) if and only if they have the same remainder when divided by h(x); that is if

f(x) mod h(x) = r(x) = p(x) mod h(x).

We denote this by

f(x) ≡ p(x) (mod h(x)).

Example 4.1.15 Let h(x) = 1 + x 5 and f(x) = 1 + x 4 + x 9 + x 11. Then dividing f(x) by h(x) gives a remainder of r(x) = 1 + x. We say that r(x) = f(x) mod h(x).

Similarly, if p(x) = 1 + x 6, then 1 + x = 1 + x 6 mod (1 + x 5) and thus we say p(x) ≡ f(x) (mod h(x)).

Example 4.1.16 Let h(x) = 1 + x 2 + x 5. Computing f(x) mod h(x), with f(x) = 1 + x 2 + x 6 + x 9 + x 11 we find that the remainder r(x) = x + x 4 and hence x + x 4 = f(x) mod h(x). Note that if p(x) = x 2 + x 8 then p(x) mod h(x) = 1 + x 3 and p(x) and f(x) are not equivalent mod h(x).

Addition and multiplication of polynomials ''respects" the equivalence of polynomials defined above. That is to say:

Lemma 4.1.17 If f(x) ≡ g(x) (mod h(x)), then

and

Proof: Suppose r(x) = f(x) mod h(x) and r(x) = g(x) mod h(x) and s(x) = p(x) mod h(x) then we have

Equivalently r(x) + s(x) = (f(x) + p(x)) mod h(x) since deg(r(x) = s(x)) < deg(h(x)). (Why?) Similar arguments show that r(x) + s(x) = (g(x) + p(x)) mod h(x). We leave the remaining arguments to Exercise 4.1.22.

< previous page | page_94 | next page >


page_95

< previous page | page_95 | next page >

Page 95

Example 4.1.18 Let h(x) = 1 + x 5, f(x) = 1 + x + x 7, g(x) = 1 + x + x 2 and p(x) = 1 + x 6; so f(x) ≡ g(x) (mod h(x)). Then

f(x) + p(x) = x + x 6 + x 7

and

g(x) + p(x) = x + x 2 + x 6

but

(x + x 6 + x 7) mod h(x) = x 2 = (x + x 2 + x 6) mod h(x).

Similarly

(1 + x + x 7)(1 + x 6) mod h(x) = 1 + x 3 = (1 + x + x 2)(1 + x 6) mod h(x).

Note that 1 + x = (1 + x 6) mod h(x). Thus we have

Exercises

4.1.19 Let h(x) = 1 + x 3 + x 5. Compute f(x) mod h(x) and its corresponding word:

(a) f(x) = 1 + x + x 6

(b) f(x) = x + x 4 + x 7 + x 8

(c) f(x) = 1 + x 10

4.1.20 Let h(x) = 1 + x 7. Compute f(x) mod h(x) and p(x) mod h(x), and decide whether f(x) ≡ p(x) (mod h(x)):

(a) f(x) = 1 + x 3 + x 8, p(x) = x + x 3 + x 7

(b) f(x) = x + x 5 + x 9, p(x) = x + x 5 + x 6 + x 13

(c) f(x) = 1 + x, p(x) = x + x 7

4.1.21 Let h(x) = 1 + x 7 compute (f(x) + g(x)) mod h(x) and (f(x)g(x)) mod h(x), where

(a) f(x) = 1 + x 6 + x 8, g(x) = 1 + x

(b) f(x) = 1 + x 5 + x 9, g(x) = x + x 2 + x 7

(c) f(x) = 1 + x 4 + x 5, g(x) = 1 + x + x 2

4.1.22 Prove that if f(x) ≡ g(x) (mod (h(x)) then f(x)p(x) ≡ g(x)p(x) (mod h(x)).
< previous page | page_95 | next page >


page_96

< previous page | page_96 | next page >

Page 96

4.2 Introduction to cyclic codes

We now begin the study of a class of codes, called cyclic codes. Eventually we will be able to use our knowledge of cyclic codes to construct a generating matrix for the two error correcting BCH codes, as well as some other codes. In fact we shall also see that the Hamming and Golay codes are cyclic codes or are equivalent to cyclic codes.

Let v be a word of length n. The cyclic shift π(v) of v is the word of length n obtained from v by taking the last digit of v and moving it to the beginning, all other digits moving one position to the right. For example:

v | 10110 | 111000 | 0000 | 1011
π(v) | 01011 | 011100 | 0000 | 1101

A code C is said to be a cyclic code if the cyclic shift of each codeword is also a codeword.

Example 4.2.1 The code C = {000, 110, 101, 011} is a linear cyclic code. First C is linear. Next we compute π(v) for all v in C.

π(000) = 000, π(110) = 011, π(101) = 110, π(011) = 101.

Since π(v) is also in C, for each v in C, C is cyclic.

Example 4.2.2 The code C = {000, 100, 011, 111} is not cyclic. The cyclic shift of v = 100 is π(100) = 010 which is not in C.

Note that the cyclic shift π is a linear transformation; that is,

Lemma 4.2.3 π(v + w) = π(v) + π(w) and π(av) = aπ(v), a ∈ K = {0, 1}. Thus to show that a linear code C is cyclic it is enough to show that π(v) π(v) ∈ C for each word v in a basis for C.

Proof: Let v = (v 0 v 1 ... v n–1), w = (w 0 w 1 ... w n–1) then v + w = (v 0 + w 0, v 1 + w 1, ..., v n–1 + w n–1) and π(v + w) = (v n–1 + w n–1, v 0 + w 0, ...., v n–2 + w n–2) = π(v) + π(w).

Example 4.2.4 In Example 4.2.1, {110, 101} is a basis for C. Since π(110) = 011 and π(101) = 110 are in C, C is a linear cyclic code.

If we wish to construct a cyclic linear code then we can pick a word, v, form a set S consisting of v and all of its cyclic shifts, S = {v, π(v), π 2(v), ..., π n–1(v)} and define C to be the linear span of S; that is C = <S>. (We use the notation π 2(v) = π(π(v)), π 3(v) = π(π(π(v))), etc.) Since S contains a basis for C, C must be cyclic by Lemma 4.2.3.

Example 4.2.5 Let n = 3 and v = 100. Then S = {v, π(v), π 2(v)} = {100, 010, 001} and <S> = K 3. Note that if w = a 0 v + a 1 π(v) + a 2 π 2(v) then π(w) = a 0 π(v) + a 1 π 2(v) + a 2 π 3(v) = a 2 v + a 0 π(v) + a 1 π 2(v).
< previous page | page_96 | next page >


page_97

< previous page | page_97 | next page >

Page 97

Example 4.2.6 Let n = 4 and v = 0101. Then π(v) = 1010 and π 2(v) = 0101 = v. Thus S = {0101, 1010} and C = <S> is the cyclic code, C = {0000, 0101, 1010, 1111}.

If a word v and its cyclic shifts form a set S = {v, π(v), ..., π n–1(v)} which spans the code C (so C = <S>), then we say v is a generator of the linear cyclic code C. Since every linear cyclic code which contains v must contain S as well, we say that C is the smallest linear cyclic code containing v. It is worth noting that a linear cyclic code can have many generators.

Exercises

4.2.7 Find a basis for the smallest linear cyclic code of length n, containing v:

(a) v = 1101000, n = 7

(b) v = 010101, n = 6

(c) v = 11011000, n = 8

4.2.8 Find all words v of length n, such that π(v) = v.

4.2.9 Find all words v of length 6 such that

(a) π 2(v) = v

(b) π 3(v) = v.

Cyclic codes have a slick representation in terms of polynomials. This is based on the simple observation that if the word v corresponds to the polynomial v(x) then the cyclic shift of v, π(v) corresponds to the polynomial xv(x) mod 1 + xn. Note that in general 1 ≡ xn (mod 1 + xn).

Example 4.2.10 Let v = 100 then v(x) = 1 and π(v) = 010 corresponds to xv(x) = x. Similarly if v = 1101 then v(x) = 1 + x + x 3 and π(v) = 1110 corresponds to xv(x) mod 1 + x 4 = 1 + x + x 2.

For cyclic codes we refer to the elements of the code both as codewords and polynomials. We can now restate the previous discussion of cyclic codes in terms of polynomials. Given a word v of length n, let the polynomial corresponding to it be v(x); then the cyclic shifts of v correspond to the polynomials xi v(x) mod 1 + xn for i = 0, 1, ..., n–1.

Example 4.2.11 Let v = 1101000 and n = 7. Then v(x) = 1 + x + x 3 and we calcuate xi v(x) for 1 ≤ i ≤ 6 in Table 4.1.

Clearly if c(x) ∈ <{v(x), xv(x), ..., x n–1 v(x)}> (reducing each product mod 1 + xn), then that means that
< previous page | page_97 | next page >


page_98

< previous page | page_98 | next page >

Page 98

word | polynomial (mod 1 + x 7)
0110100 | xv(x) = x + x 2 + x 4
0011010 | x 2 v(x) = x 2 + x 3 + x 5
0001101 | x 3 v(x) = x 3 + x 4 + x 6
1000110 | x 4 v(x) = x 4 + x 5 + x 7 ≡ 1 + x 4 + x 5 (mod 1 + x 7)
0100011 | x 5 v(x) = x 5 + x 6 + x 8 ≡ x + x 5 + x 6 (mod 1 + x 7)
1010001 | x 6 v(x) = x 6 + x 7 + x 9 ≡ 1 + x 2 + x 6 (mod 1 + x 7)
Table 4.1: Polynomial representation of cyclic shifts.

Therefore we obtain the following result.

Lemma 4.2.12 Let C be a cyclic code and let v ∈ C. Then for any polynomial a(x), c(x) = a(x)v(x) mod (1 + xn) is a codeword in C.

Among all non-zero codewords in a linear cyclic code C, there is a unique word g ∈ C, such g(x) has minimum degree, as the following argument indicates. Certainly there is at least one word or polynomial of smallest degree in C. If two non-zero words g and g′ correspond to polynomials g(x) and g′(x) of minimum degree k then g(x) + g′(x) = c(x) ∈ C since C is linear and degree (c(x)) < k (since x k + x k = 0). Since g is a non-zero word of smallest degree, degree (c(x)) < k means that c(x) = 0, so g(x) = g′(x) and so g(x) is unique.

We define the generator polynomial of a linear cyclic code C to be the unique non-zero polynomial of minimum degree in C. From the above discussion we know it is unique, but is it a generator?

To see that it is, we must show that for any codeword c(x) ∈ C, there exists a(x) such that, c(x) = a(x)g(x) mod 1 + xn; in fact we shall show that c(x) = a(x)g(x). Since deg(c(x)) ≥ deg(g(x)) we have by the Division Algorithm,

c(x) = q(x)g(x) + r(x)

or

r(x) = q(x)g(x) + c(x).

However both c(x) and q(x)g(x) are codewords of C by Lemma 4.2.12 and thus so is r(x). But by the Division Algorithm either r(x) = 0 or deg(r(x)) < deg(g(x)). Since the latter is impossible unless r = 0, we conclude that r(x) = 0 and thus g(x) is a divisor of every codeword c(x) in C.

Theorem 4.2.13 Let C be a cyclic code of length n and let g(x) be the generator polynomial. If n – k = degree (g(x)) then

1. C has dimension k,

2. the codewords corresponding to g(x), xg(x), ..., x k–1 g(x) are a basis for C, and
< previous page | page_98 | next page >


page_99

< previous page | page_99 | next page >

Page 99

3. c(x) ∈ C if and only if c(x) = a(x)g(x) for some polynomial a(x) with degree (a(x)) < k (that is, g(x) is a divisor of every codeword c(x)).

Proof: The discussion before Theorem 4.2.13 proves (3). If g(x) has degree n – k then g(x), xg(x), ..., x k–1 g(x) must be linearly independent. (Why?) Since g(x) divides every codeword there is a unique polynomial a(x) = a 0 + a 1 x + · · · + a k–1 x k–1 such that c(x) = a(x)g(x) = a 0 g(x) + a 1 xg(x) + · · · + a k–1 x k–1 g(x). Therefore c(x) is in <{g(x), xg(x), ..., x k–1 g(x)}> and thus {g(x), xg(x), ..., x k–1 g(x)} is a basis for C.

Example 4.2.14 Let n = 7, g(x) = 1 + x + x 3 be the generator for the cyclic code C. A basis for C is

Note x 4 g(x) mod 1 + x 7 = 1 + x 4 + x 5 is a codeword since 1 + x 4 + x 5 = (1 + x + x 2)(1 + x + x 3) = (1 + x + x 2)g(x).

Example 4.2.15 Let C be the cyclic code C = {0000, 1010, 0101, 1111}; the corresponding polynomials are {0, 1 + x 2, 1 + x + x 2 + x 3}. Note, 1 + x 2 ↔ 1010 is the generator polynomial for C, since C contains only one polynomial of degree 2 and none of degree 1. Also, every word (polynomial) in C is a multiple of the generator polynomial:

0 = 0(1 + x 2) x + x 3 = x(1 + x 2)

Example 4.2.16 The smallest linear cyclic code C of length 6 containing g(x) = 1 + x 3 ↔ 100100 is

= {000000, 100100, 010010, 001001, 110110, 101101, 011011, 111111}.

This can be verified by the techniques described earlier in this section. The polynomial of smallest degree representing a word in C is seen by inspection to be g(x) = 1 + x 3, and C contains no other polynomial of degree 3. Thus g(x) = 1 + x 3 is the generator polynomial for C. We represent each word in C as a multiple of g(x) (see Table 4.2).

We can generate cyclic codes easily enough by picking a word v and setting C = <{v(x), xv(x), ..., x n–1 v(x)}> (modulo 1 + xn). However we need to find the generator polynomial for such a code and listing all codewords is not a reasonable approach. The generator polynomial for a cyclic code has one important property:
< previous page | page_99 | next page >


page_f1

< previous page | page_f1 | next page >
Page F1 CODING THEORY AND CRYPTOGRAPHY THE ESSENTIALS
< previous page | page_f1 | next page >


page_f2

< previous page | page_f2 | next page >

Page F2

PURE AND APPLIED MATHEMATICS

A Program of Monographs, Textbooks, and Lecture Notes

EXECUTIVE EDITORS

Earl J. Taft
Rutgers University
New Brunswick, New Jersey

Zuhair Nashed
University of Delaware
Newark, Delaware

EDITORIAL BOARD

M. S. Baouendi
University of California,
San Diego

Jane Cronin
Rutgers University

Jack K. Hale
Georgia Institute of Technology

S. Kobayashi
University of California,
Berkeley

Marvin Marcus
University of California,
Santa Barbara

W. S. Massey
Yale University

Anil Nerode
Cornell University

Donald Passman
University of Wisconsin,
Madison

Fred S. Roberts
Rutgers University

David L. Russell
Virginia Polytechnic Institute
and State University

Walter Schempp
Universität Siegen

Mark Teply
University of Wisconsin,
Milwaukee
< previous page | page_f2 | next page >


page_f3

< previous page | page_f3 | next page >

Page F3

MONOGRAPHS AND TEXTBOOKS IN PURE AND APPLIED MATHEMATICS

1. K. Yano, Integral Formulas in Riemannian Geometry (1970)

2. S. Kobayashi, Hyperbolic Manifolds and Holomorphic Mappings (1970)

3. V. S. Vladimirov, Equations of Mathematical Physics (A. Jeffrey, ed.; A. Littlewood, trans.) (1970)

4. B. N. Pshenichnyi, Necessary Conditions for an Extremum (L. Neustadt, translation ed.; K. Makowski, trans.) (1971)

5. L. Narici et al., Functional Analysis and Valuation Theory (1971)

6. S. S. Passman, Infinite Group Rings (1971)

7. L. Domhoff, Group Representation Theory. Part A: Ordinary Representation Theory. Part B: Modular Representation Theory (1971, 1972)

8. W. Boothby and G. L. Weiss, eds., Symmetric Spaces (1972)

9. Y. Matsushima, Differentiable Manifolds (E. T. Kobayashi, trans.) (1972)

10. L. E. Ward, Jr., Topology (1972)

11. A. Babakhanian, Cohomological Methods in Group Theory (1972)

12. R. Gilmer, Multiplicative Ideal Theory (1972)

13. J. Yeh, Stochastic Processes and the Wiener Integral (1973)

14. J. Barros-Neto, Introduction to the Theory of Distributions (1973)

15. R. Larsen, Functional Analysis (1973)

16. K. Yano and S. Ishihara, Tangent and Cotangent Bundles (1973)

17. C. Procesi, Rings with Polynomial Identities (1973)

18. R. Hermann, Geometry, Physics, and Systems (1973)

19. N. R. Wallach, Harmonic Analysis on Homogeneous Spaces (1973)

20. J. Dieudonné, Introduction to the Theory of Formal Groups (1973)

21. I. Vaisman, Cohomology and Differential Forms (1973)

22. B.-Y. Chen, Geometry of Submanifolds (1973)

23. M. Marcus, Finite Dimensional Multilinear Algebra (in two parts) (1973, 1975)

24. R. Larsen, Banach Algebras (1973)

25. R. O. Kujala and A. L. Vitter, eds., Value Distribution Theory: Part A; Part B: Deficit and Bezout Estimates by Wilhelm Stoll (1973)

26. K. B. Stolarsky, Algebraic Numbers and Diophantine Approximation (1974)

27. A. R. Magid, The Separable Galois Theory of Commutative Rings (1974)

28. B. R. McDonald, Finite Rings with Identity (1974)

29. J. Satake, Linear Algebra (S. Koh et al., trans.) (1975)

30. J. S. Golan, Localization of Noncommutative Rings (1975)

31. G. Klambauer, Mathematical Analysis (1975)

32. M. K. Agoston, Algebraic Topology (1976)

33. K. R. Goodearl, Ring Theory (1976)

34. L. E. Mansfield, Linear Algebra with Geometric Applications (1976)

35. N. J. Pullman, Matrix Theory and Its Applications (1976)

36. B. R. McDonald, Geometric Algebra Over Local Rings (1976)

37. C. W. Groetsch, Generalized Inverses of Linear Operators (1977)

38. J. E. Kuczkowski and J. L. Gersting, Abstract Algebra (1977)

39. C. O. Christenson and W. L. Voxman, Aspects of Topology (1977)

40. M. Nagata, Field Theory (1977)

41. R. L. Long, Algebraic Number Theory (1977)

42. W. F. Pfeffer, Integrals and Measures (1977)

43. R. L. Wheeden and A. Zygmund, Measure and Integral (1977)

44. J. H. Curtiss, Introduction to Functions of a Complex Variable (1978)

45. K. Hrbacek and T. Jech, Introduction to Set Theory (1978)

46. W. S. Massey, Homology and Cohomology Theory (1978)

47. M. Marcus, Introduction to Modern Algebra (1978)

48. E. C. Young, Vector and Tensor Analysis (1978)

49. S. B. Nadler, Jr., Hyperspaces of Sets (1978)

50. S. K. Segal, Topics in Group Kings (1978)

51. A. C. M. van Rooij, Non-Archimedean Functional Analysis (1978)

52. L. Corwin and R. Szczarba, Calculus in Vector Spaces (1979)

53. C. Sadosky, Interpolation of Operators and Singular Integrals (1979)

54. J. Cronin, Differential Equations (1980)

55. C. W. Groetsch, Elements of Applicable Functional Analysis (1980)
< previous page | page_f3 | next page >


page_f4

< previous page | page_f4 | next page >

Page F4

56. I. Vaisman, Foundations of Three-Dimensional Euclidean Geometry (1980)

57. H. I. Freedan, Deterministic Mathematical Models in Population Ecology (1980)

58. S. B. Chae, Lebesgue Integration (1980)

59. C. S. Rees et al., Theory and Applications of Fourier Analysis (1981)

60. L. Nachbin, Introduction to Functional Analysis (R. M. Aron, trans.) (1981)

61. G. Orzech and M. Orzech, Plane Algebraic Curves (1981)

62. R. Johnsonbaugh and W. E. Pfaffenberger, Foundations of Mathematical Analysis (1981)

63. W. L. Voxman and R. H. Goetschel, Advanced Calculus (1981)

64. L. J. Corwin and R. H. Szczarba, Multivariable Calculus (1982)

65. V. I. Istr[acaron]tescu, Introduction to Linear Operator Theory (1981)

66. R. D. Järvinen, Finite and Infinite Dimensional Linear Spaces (1981)

67. J. K. Beem and P. E. Ehrlich, Global Lorentzian Geometry (1981)

68. D. L. Armacost, The Structure of Locally Compact Abelian Groups (1981)

69. J. W. Brewer and M. K. Smith, eds., Emmy Noether: A Tribute (1981)

70. K. H. Kim, Boolean Matrix Theory and Applications (1982)

71. T. W. Wieting, The Mathematical Theory of Chromatic Plane Omaments (1982)

72. D. B. Gauld, Differential Topology (1982)

73. R. L. Faber, Foundations of Euclidean and Non-Euclidean Geometry (1983)

74. M. Carmeli, Statistical Theory and Random Matrices (1983)

75. J. H. Carruth et al., The Theory of Topological Semigroups (1983)

76. R. L. Faber, Differential Geometry and Relativity Theory (1983)

77. S. Bamett, Polynomials and Linear Control Systems (1983)

78. G. Karpilovsky, Commutative Group Algebras (1983)

79. F. Van Oystaeyen and A. Verschoren, Relative Invariants of Rings (1983)

80. I. Vaisman, A First Course in Differential Geometry (1984)

81. G. W. Swan, Applications of Optimal Control Theory in Biomedicine (1984)

82. T. Petrie and J. D. Randall, Transformation Groups on Manifolds (1984)

83. K. Goebel and S. Reich, Uniform Convexity, Hyperbolic Geometry, and Nonexpansive Mappings (1984)

84. T. Albu and C. N[acaron]st[acaron]sescu, Relative Finiteness in Module Theory (1984)

85. K. Hrbacek and T. Jech, Introduction to Set Theory: Second Edition (1984)

86. F. Van Oystaeyen and A. Verschoren, Relative Invariants of Rings (1984)

87. B. R. McDonald, Linear Algebra Over Commutative Rings (1984)

88. M. Namba, Geometry of Projective Algebraic Curves (1984)

89. G. F. Webb, Theory of Nonlinear Age-Dependent Population Dynamics (1985)

90. M. R. Bremner et al., Tables of Dominant Weight Multiplicities for Representations of Simple Lie Algebras (1985)

91. A. E. Fekete, Real Linear Algebra (1985)

92. S. B. Chae, Holomorphy and Calculus in Normed Spaces (1985)

93. A. J. Jerri, Introduction to Integral Equations with Applications (1985)

94. G. Karpilovsky, Projective Representations of Finite Groups (1985)

95. L. Narici and E. Beckenstein, Topological Vector Spaces (1985)

96. J. Weeks, The Shape of Space (1985)

97. P. R. Gribik and K. O. Kortanek, Extremal Methods of Operations Research (1985)

98. J.-A. Chao and W. A. Woyczynski, eds., Probability Theory and Harmonic Analysis (1986)

99. G. D. Crown et al., Abstract Algebra (1986)

100. J. H. Carruth et al., The Theory of Topological Semigroups, Volume 2 (1986)

101. R. S. Doran and V. A. Belfi, Characterizations of C*-Algebras (1986)

102. M. W. Jeter, Mathematical Programming (1986)

103. M. Altman, A Unified Theory of Nonlinear Operator and Evolution Equations with Applications (1986)

104. A. Verschoren, Relative Invariants of Sheaves (1987)

105. R. A. Usmani, Applied Linear Algebra (1987)

106. P. Blass and J. Lang, Zariski Surfaces and Differential Equations in Characteristic ρ > 0 (1987)

107. J. A. Reneke et al., Structured Hereditary Systems (1987)

108. H. Busemann and B. B. Phadke, Spaces with Distinguished Geodesics (1987)

109. R. Harte, Invertibility and Singularity for Bounded Linear Operators (1988)

110. G. S. Ladde et al., Oscillation Theory of Differential Equations with Deviating Arguments (1987)

111. L. Dudkin et al., Iterative Aggregation Theory (1987)

112. T. Okubo, Differential Geometry (1987)
< previous page | page_f4 | next page >


page_f5

< previous page | page_f5 | next page >

Page F5

113. D. L. Stancl and M. L. Stancl, Real Analysis with Point-Set Topology (1987)

114. T. C. Gard, Introduction to Stochastic Differential Equations (1988)

115. S. S. Abhyankar, Enumerative Combinatorics of Young Tableaux (1988)

116. H. Strade and R. Famsteiner, Modular Lie Algebras and Their Representations (1988)

117. J. A. Huckaba, Commutative Rings with Zero Divisors (1988)

118. W. D. Wallis, Combinatorial Designs (1988)

119. W. Wieslaw, Topological Fields (1988)

120. G. Karpilovsky, Field Theory (1988)

121. S. Caenepeel and F. Van Oystaeyen, Brauer Groups and the Cohomology of Graded Rings (1989)

122. W. Kozlowski, Modular Function Spaces (1988)

123. E. Lowen-Colebunders, Function Classes of Cauchy Continuous Maps (1989)

124. M. Pavel, Fundamentals of Pattern Recognition (1989)

125. V. Lakshmikantham et al., Stability Analysis of Nonlinear Systems (1989)

126. R. Sivaramakrishnan, The Classical Theory of Arithmetic Functions (1989)

127. N. A. Watson, Parabolic Equations on an Infinite Strip (1989)

128. K. J. Hastings, Introduction to the Mathematics of Operations Research (1989)

129. B. Fine, Algebraic Theory of the Bianchi Groups (1989)

130. D. N. Dikranjan et al., Topological Groups (1989)

131. J. C. Morgan II, Point Set Theory (1990)

132. P. Biler and A. Witkowski, Problems in Mathematical Analysis (1990)

133. H. J. Sussmann, Nonlinear Controllability and Optimal Control (1990)

134. J.-P. Florens et al., Elements of Bayesian Statistics (1990)

135. N. Shell, Topological Fields and Near Valuations (1990)

136. B. F. Doolin and C. F. Martin, Introduction to Differential Geometry for Engineers (1990)

137. S. S. Holland, Jr., Applied Analysis by the Hilbert Space Method (1990)

138. J. Okninski, Semigroup Algebras (1990)

139. K. Zhu, Operator Theory in Function Spaces (1990)

140. G. B. Price, An Introduction to Multicomplex Spaces and Functions (1991)

141. R. B. Darst, Introduction to Linear Programming (1991)

142. P. L. Sachdev, Nonlinear Ordinary Differential Equations and Their Applications (1991)

143. T. Husain, Orthogonal Schauder Bases (1991)

144. J. Foran, Fundamentals of Real Analysis (1991)

145. W. C. Brown, Matrices and Vector Spaces (1991)

146. M. M. Rao and Z. D. Ren, Theory of Orlicz Spaces (1991)

147. J. S. Golan and T. Head, Modules and the Structures of Rings (1991)

148. C. Small, Arithmetic of Finite Fields (1991)

149. K. Yang, Complex Algebraic Geometry (1991)

150. D. G. Hoffman et al., Coding Theory (1991)

151. M. O. González, Classical Complex Analysis (1992)

152. M. O. González, Complex Analysis (1992)

153. L. W. Baggett, Functional Analysis (1992)

154. M. Sniedovich, Dynamic Programming (1992)

155. R. P. Agarwal, Difference Equations and Inequalities (1992)

156. C. Brezinski, Biorthogonality and Its Applications to Numerical Analysis (1992)

157. C. Swartz, An Introduction to Functional Analysis (1992)

158. S. B. Nadler, Jr., Continuum Theory (1992)

159. M. A. Al-Gwaiz, Theory of Distributions (1992)

160. E. Perry, Geometry: Axiomatic Developments with Problem Solving (1992)

161. E. Castillo and M. R. Ruiz-Cobo, Functional Equations and Modelling in Science and Engineering (1992)

162. A. J. Jerri, Integral and Discrete Transforms with Applications and Error Analysis (1992)

163. A. Charlier et al., Tensors and the Clifford Algebra (1992)

164. P. Biler and T. Nadzieja, Problems and Examples in Differential Equations (1992)

165. E. Hansen, Global Optimization Using Interval Analysis (1992)

166. S. Guerre-Delabrière, Classical Sequences in Banach Spaces (1992)

167. Y. C. Wong, Introductory Theory of Topological Vector Spaces (1992)

168. S. H. Kulkami and B. V. Limaye, Real Function Algebras (1992)

169. W. C. Brown, Matrices Over Commutative Rings (1993)

170. J. Loustau and M. Dillon, Linear Geometry with Computer Graphics (1993)

171. W. V. Petryshyn, Approximation-Solvability of Nonlinear Functional and Differential Equations (1993)
< previous page | page_f5 | next page >


page_f6

< previous page | page_f6 | next page >

Page F6

172. E. C. Young, Vector and Tensor Analysis: Second Edition (1993)

173. T. A. Bick, Elementary Boundary Value Problems (1993)

174. M. Pavel, Fundamentals of Pattern Recognition: Second Edition (1993)

175. S. A. Albeverio et al., Noncommutative Distributions (1993)

176. W. Fulks, Complex Variables (1993)

177. M. M. Rao, Conditional Measures and Applications (1993)

178. A. Janicki and A. Weron, Simulation and Chaotic Behavior of α-Stable Stochastic Processes (1994)

179. P. Neittaanmäki and D. Tiba, Optimal Control of Nonlinear Parabolic Systems (1994)

180. J. Cronin, Differential Equations: Introduction and Qualitative Theory, Second Edition (1994)

181. S. Heikkilä and V. Lakshmikantham, Monotone Iterative Techniques for Discontinuous Nonlinear Differential Equations (1994)

182. X. Mao, Exponential Stability of Stochastic Differential Equations (1994)

183. B. S. Thomson, Symmetric Properties of Real Functions (1994)

184. J. E. Rubio, Optimization and Nonstandard Analysis (1994)

185. J. L. Bueso et al., Compatibility, Stability, and Sheaves (1995)

186. A. N. Michel and K. Wang, Qualitative Theory of Dynamical Systems (1995)

187. M. R. Damel, Theory of Lattice-Ordered Groups (1995)

188. Z. Naniewicz and P. D. Panagiotopoulos, Mathematical Theory of Hemivariational Inequalities and Applications (1995)

189. L. J. Corwin and R. H. Szczarba, Calculus in Vector Spaces: Second Edition (1995)

190. L. H. Erbe et al., Oscillation Theory for Functional Differential Equations (1995)

191. S. Agaian et al., Binary Polynomial Transforms and Nonlinear Digital Filters (1995)

192. M. I. Gil', Norm Estimations for Operation-Valued Functions and Applications (1995)

193. P. A. Grillet, Semigroups: An Introduction to the Structure Theory (1995)

194. S. Kichenassamy, Nonlinear Wave Equations (1996)

195. V. F. Krotov, Global Methods in Optimal Control Theory (1996)

196. K. I. Beidar et al., Rings with Generalized Identities (1996)

197. V. I. Amautov et al., Introduction to the Theory of Topological Rings and Modules (1996)

198. G. Sierksma, Linear and Integer Programming (1996)

199. R. Lasser, Introduction to Fourier Series (1996)

200. V. Sima, Algorithms for Linear-Quadratic Optimization (1996)

201. D. Redmond, Number Theory (1996)

202. J. K. Beem et al., Global Lorentzian Geometry: Second Edition (1996)

203. M. Fontana et al., Prüfer Domains (1997)

204. H. Tanabe, Functional Analytic Methods for Partial Differential Equations (1997)

205. C. Q. Zhang, Integer Flows and Cycle Covers of Graphs (1997)

206. E. Spiegel and C. J. O'Donnell, Incidence Algebras (1997)

207. B. Jakubczyk and W. Respondek, Geometry of Feedback and Optimal Control (1998)

208. T. W. Haynes et al., Fundamentals of Domination in Graphs (1998)

209. T. W. Haynes et al., Domination in Graphs: Advanced Topics (1998)

210. L. A. D'Alotto et al., A Unified Signal Algebra Approach to Two-Dimensional Parallel Digital Signal Processing (1998)

211. F. Halter-Koch, Ideal Systems (1998)

212. N. K. Govil et al., Approximation Theory (1998)

213. R. Cross, Multivalued Linear Operators (1998)

214. A. A. Martynyuk, Stability by Liapunov's Matrix Function Method with Applications (1998)

215. A. Favini and A. Yagi, Degenerate Differential Equations in Banach Spaces (1999)

216. A. Illanes and S. Nadler, Jr., Hyperspaces: Fundamentals and Recent Advances (1999)

217. G. Kato and D. Struppa, Fundamentals of Algebraic Microlocal Analysis (1999)

218. G. X.-Z. Yuan, KKM Theory and Applications in Nonlinear Analysis (1999)

219. D. Motreanu and N. H. Pavel, Tangency, Flow Invariance for Differential Equations, and Optimization Problems (1999)

220. K. Hrbacek and T. Jech, Introduction to Set Theory, Third Edition (1999)

221. G. E. Kolosov, Optimal Design of Control Systems (1999)

222. N. L. Johnson, Subplane Covered Nets (2000)

223. B. Fine and G. Rosenberger, Algebraic Generalizations of Discrete Groups (1999)

224. M. Väth, Volterra and Integral Equations of Vector Functions (2000)

225. S. S. Miller and P. T. Mocanu, Differential Subordinations (2000)
< previous page | page_f6 | next page >


page_f7

< previous page | page_f7 | next page >

Page F7

226. R. Li et al., Generalized Difference Methods for Differential Equations: Numerical Analysis of Finite Volume Methods (2000)

227. H. Li and F. Van Oystaeyen, A Primer of Algebraic Geometry (2000)

228. R. P. Agarwal, Difference Equations and Inequalities: Theory, Methods, and Applications, Second Edition (2000)

229. A. B. Kharazishvili, Strange Functions in Real Analysis (2000)

230. J. M. Appell et al., Partial Integral Operators and Integro-Differential Equations (2000)

231. A. I. Prilepko et al., Methods for Solving Inverse Problems in Mathematical Physics (2000)

232. F. Van Oystaeyen, Algebraic Geometry for Associative Algebras (2000)

233. D. Jagerman, Difference Equations with Applications to Queues (2000)

234. D. R. Hankerson, D. G. Hoffman, D. A. Leonard, C.C. Lindner, K.T. Phelps, C. A. Rodger, J. R. Wall Coding Theory and Cryptography: The Essentials, Second Edition, Revised and Expanded (2000)

235. S. Dascalescu et al. Hopf Algebras: An Introduction (2000)

236. Hagen et al. C*-Algebras and Numerical Analysis (2000)

237. Y. Talpaert, Differential Geometry: With Applications to Mechanics and Physics (2000)

Additional Volumes in Preparation

A. Samarskii, The Theory of Difference Schemes
< previous page | page_f7 | next page >


page_ii

< previous page | page_ii | next page >

Page ii

CODING THEORY AND CRYPTOGRAPHY THE ESSENTIALS
Second Edition, Revised and Expanded

D. R. Hankerson
D. G. Hoffman
D. A. Leonard
C. C. Lindner
K. T. Phelps
C. A. Rodger
J. R. Wall

Auburn University
Auburn, Alabama
< previous page | page_ii | next page >


page_iii

< previous page | page_iii | next page >

Page iii

First edition: D. G. Hoffman, D. A. Leonard, C. C. Lindner, K. T. Phelps, C. A. Rodger, J. R. Wall, Coding Theory: The Essentials. New York: Marcel Dekker, 1991.

ISBN: 0-8247-0465-7

This book is printed on acid-free paper.

Headquarters
Marcel Dekker, Inc.
270 Madison Avenue, New York, NY 10016
tel: 212-696-9000; fax: 212-685-4540

Eastern Hemisphere Distribution
Marcel Dekker AG
Hutgasse 4, Postfach 812, CH-4001 Basel, Switzerland
tel: 41-61-261-8482; fax: 41-61-261-8896

World Wide Web
http://www.dekker.com

The publisher offers discounts on this book when ordered in bulk quantities. For more information, write to Special Sales/Professional Marketing at the headquarters address above.

Copyright © 2000 by Marcel Dekker, Inc. All Rights Reserved.

Neither this book nor any part may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, microfilming, and recording, or by any information storage and retrieval system, without permission in writing from the publisher.

Current printing (last digit):
10 9 8 7 6 5 4 3

PRINTED IN THE UNITED STATES OF AMERICA
< previous page | page_iii | next page >


page_iv

< previous page | page_iv | next page >
Page iv To our wonderful spouses Cindy, Gail, Jane, Ann, Janet, and Sue Our children Noel, Ian, Tim, Curt, Jimmy, Andrew, Meghan, Katrina, and Rebecca And our parents Eileen and Richard, Vally and Gale, Marjorie and Lewis, Mary and Charles Ethel and Richard, Iris and Ian, and Beulah and Walter
< previous page | page_iv | next page >


page_ix

< previous page | page_ix | next page >

Page ix

9 Reed-Muller and Preparata Codes | 205
9.1 Reed-Muller codes | 205
9.2 Decoding Reed-Muller codes | 208
9.3 Extended Preparata codes | 213
9.4 Encoding extended Preparata codes | 219
9.5 Decoding extended Preparata codes | 222
Part II: Cryptography
10 Classical Cryptography | 227
10.1 Encryption schemes | 228
10.2 Symmetric-key encryption | 230
10.3 Feistel ciphers and DES | 238
10.3.1 The New Data Seal | 240
10.3.2 The Data Encryption Standard | 243
10.4 Notes | 249
11 Topics in Algebra and Number Theory | 253
11.1 Algorithms, complexity, and modular arithmetic | 253
11.2 Quadratic residues | 260
11.3 Primality testing | 264
11.4 Factoring and square roots | 267
11.4.1 Pollard's rho | 267
11.4.2 Random squares | 269
11.4.3 Square roots | 271
11.5 Discrete logarithms | 274
11.5.1 Baby-step giant-step | 274
11.5.2 Index calculus | 275
11.6 Notes | 277
12 Public-key Cryptography | 279
12.1 One-way and hash functions | 280
12.2 RSA | 284
12.3 Provable security | 290
12.4 ElGamal | 293
12.5 Cryptographic protocols | 297
12.5.1 Diffie-Hellman key agreement | 298
12.5.2 Zero-knowledge proofs | 299
12.5.3 Coin-tossing and mental poker | 301
12.6 Notes | 305
A The Euclidean Algorithm | 307
< previous page | page_ix | next page >


page_v

< previous page | page_v | next page >

Page v

Preface

This book, revised and updated from the first edition, is designed to teach coding theory and cryptography in a mathematically sound manner to students in engineering, computer science, and mathematics. It differs from most other texts on the subject in two important ways: the ''just in time philosophy," and unnecessary mathematical generalizations are omitted.

The "just in time" philosophy consists of introducing the necessary mathematics just in time to be applied; i.e., juxtaposed, with the applications. We don't have 200 pages of mathematics (most of which is irrelevant) followed by 200 pages of coding theory and cryptography. So the format is roughly: mathematics, applications, mathematics, applications, etc. Avoiding unnecessary generalizations means that we don't find it necessary, for example, to describe a cyclic code as a principal ideal. In other words, we have for the most part omitted the mathematical generalizations and terminology that would normally be used in teaching a course to a class consisting entirely of advanced mathematics majors.

Part I (Chapters 1–9) of this text has been used to teach a two-semester sequence in coding theory at Auburn University. The minimal prerequisite for students taking this course is a rather elementary knowledge of linear algebra. However, the more linear algebra, as well as general modern algebra, students bring to the course the better. Students with more mathematical background and maturity will be able to move rather quickly through the early material.

The coding theory portion deals exclusively with binary codes and codes over fields of characteristic 2, stressing the construction, encoding and decoding of several important families of codes. Primarily, we have chosen families of codes that are of interest in engineering and computer science, such as Reed-Solomon codes and convolutional codes, which have been used in deep space communications and consumer electronics (to name but two areas of application). This choice of codes also reflects a broad range of algorithms for encoding and decoding.

Part II (Chapters 10–12) has its origins in an introductory semester-length course on cryptography taught at Auburn University. The course attracts a diverse audience of graduate and undergraduate students from computer science, engineering, education, and mathematics, some of whom will have had only an introductory course in algebra or number theory at the sophomore level. Fortunately, this level of sophistication is sufficient to develop a respectable course

< previous page | page_v | next page >


page_vi

< previous page | page_vi | next page >

Page vi

in cryptography—indeed, most of the material here requires only a review (included in Chapter 11) of basic results concerning the integers modulo n. The intent has been to write a concise and self-contained introduction to modern cryptography, with an emphasis on public-key methods. In Chapter 12 especially, the main points are covered in relatively short sections, with additional topics outlined (usually in some detail and with references) in the exercises.

In a broad sense, coding theory and cryptography are both concerned with the electronic transfer of information—one with reliability, the other with security. We recognize that not every degree program has the luxury of including courses devoted to each topic, so this text has been written to accommodate several course designs. In a single semester, one could choose to investigate coding theory in depth, covering Chapters 1 to 4 and then either Chapters 5 and 6 or Chapters 7 and 8. One could also delve into cryptography by covering just Chapters 10–12. It is also possible to gain some knowledge in each of the two areas in one semester, in which case we recommend that Chapters 1–3, 10, and 12 be covered, with topics selected from Chapter 11 as needed.

The authors would very much appreciate any comments that users of this text care to pass along. Our email address is rodgec1@auburn.edu.

Acknowledgments

We are deeply indebted to Alfred Menezes for providing detailed suggestions and reviews of several drafts of Chapters 10–12. Without his advice, the material would have contained more errors and would have been more poorly organized. We also wish to acknowledge and thank Selda [image] for her reviews, suggestions, and corrections.

Rosie Torbert did exceptional work in creating the sources for the first edition of this book. Her never failing good cheer in enduring the slings and arrows of constant revisions places her in the saint category. We'd like to thank Heather Conner for doing such a great job in preparing this second edition. We especially appreciate the cover designs of Cindy Otterson and thank her for her work with us on several projects.

D. Hankerson, D.G. Hoffman, D.A. Leonard
C.C. Lindner, K.T. Phelps
C.A. Rodger, J.R. Wall
< previous page | page_vi | next page >


page_vii

< previous page | page_vii | next page >

Page vii

Contents

Preface | v
Part I: Coding Theory
1 Introduction to Coding Theory | 1
1.1 Introduction | 1
1.2 Basic assumptions | 2
1.3 Correcting and detecting error patterns | 4
1.4 Information rate | 6
1.5 The effects of error correction and detection | 7
1.6 Finding the most likely codeword transmitted | 8
1.7 Some basic algebra | 10
1.8 Weight and distance | 11
1.9 Maximum likelihood decoding | 12
1.10 Reliability of MLD | 16
1.11 Error-detecting codes | 18
1.12 Error-correcting codes | 22
2 Linear Codes | 27
2.1 Linear codes | 27
2.2 Two important subspaces | 28
2.3 Independence, basis, dimension | 30
2.4 Matrices | 35
2.5 Bases for C = 〈S〉 and C⊥ | 37
2.6 Generating matrices and encoding | 41
2.7 Parity-check matrices | 45
2.8 Equivalent codes | 48
2.9 Distance of a linear code | 52
2.10 Cosets | 53
2.11 MLD for linear codes | 57
2.12 Reliability of IMLD for linear codes | 62
< previous page | page_vii | next page >


page_viii

< previous page | page_viii | next page >

Page viii

3 Perfect and Related Codes | 65
3.1 Some bounds for codes | 65
3.2 Perfect codes | 70
3.3 Hamming codes | 72
3.4 Extended codes | 75
3.5 The extended Golay code | 77
3.6 Decoding the extended Golay code | 79
3.7 The Golay code | 82
3.8 Reed-Muller codes | 84
3.9 Fast decoding for RM(1,m) | 88
4 Cyclic Linear Codes | 91
4.1 Polynomials and words | 91
4.2 Introduction to cyclic codes | 96
4.3 Generating and parity check matrices for cyclic codes | 101
4.4 Finding cyclic codes | 104
4.5 Dual cyclic codes | 109
5 BCH Codes | 111
5.1 Finite fields | 111
5.2 Minimal polynomials | 115
5.3 Cyclic Hamming codes | 118
5.4 BCH codes | 120
5.5 Decoding 2 error-correcting BCH code | 122
6 Reed-Solomon Codes | 127
6.1 Codes over GF(2r) | 127
6.2 Reed-Solomon codes | 129
6.3 Decoding Reed-Solomon codes | 135
6.4 Transform approach to Reed-Solomon codes | 141
6.5 Berlekamp-Massey algorithm | 148
6.6 Erasures | 153
7 Burst Error-Correcting Codes | 159
7.1 Introduction | 159
7.2 Interleaving | 163
7.3 Application to compact discs | 170
8 Convolutional Codes | 173
8.1 Shift registers and polynomials | 173
8.2 Encoding convolutional codes | 179
8.3 Decoding convolutional codes | 186
8.4 Truncated Viterbi decoding | 193
< previous page | page_viii | next page >


page_x

< previous page | page_x | next page >

Page x

B Factorization of 1 + xn | 311
C Example of Compact Disc Encoding | 313
D Solutions to Selected Exercises | 317
Bibliography | 335
Index | 343
< previous page | page_x | next page >


Easy CHM - Create your first CHM help or CHM ebook files in 10 seconds with Easy CHM

This CHM was compiled by an UNREGISTERED version of Easy CHM.
Registered version doesn't compile this page into CHM!

Easy CHM Written by: GuohuaSoft |  | Screen Shot
 | File Size: 2989k - 2.84MB | Type: Shareware
Keywords: make chm, create chm, chm converter, html2chm, htm2chm, txt2chm, generate chm, chm maker, convert chm, create Microsoft Htmlhelp, generate Html help, help file generator, chm creation tool, create compressed html-help

Description:
Create your first CHM help or CHM ebook files in 10 seconds with Easy CHM.

Key features:

  • Make CHM (Html Help) file by importing folders and all sub folders in a fast speed.
  • Automatically generate TOC and Index Items by parsing from Html title/file name/text of a specific line.
  • Easy to use TOC editor and support 'Search and replace in TOC items', etc.
  • Fully Multi-Selection support, it's very convenient when Moving/Replacing/Deleting/Drag-Dropping Topics and Batch change icons, etc.
  • Easy CHM can drag-drop or move multiple topics at a time, and there're many handy options like 'Move UP/Down', 'Move Top/Bottom' and 'Move Sibling UP/Down' for you to choose.
  • Fully Keyboard support.
  • With the internal fast Virtual Tree, it's very fast and stable when editing a very large TOC tree.
  • Convert HTM, HTML, MHT, Picture and other files to CHM.
  • Automatically create a Context-Sensitive Html Help.
  • Automatically output Alias and Map files for developers to use.
  • EasyCHM can save/load project file, it's easy for the content developers for documentation to manage or edit their documents.
  • Convert any CHM files to html freely.
  • Plenty of customized options.
  • Multilanguage support, the interface of the program can be changed to Arabic, English, French, German, Korean, Polish, Portuguese, Russian, Simplified Chinese, Spanish, Swedish, Traditional Chinese, Turkish, Ukrainian.
  • Built-in CHM files decompiler.
  • Easy-to-use interface.
Contact Info:

Via Web:

Zipghost website: http://www.zipghost.com

eTextWizard website: http://www.etextwizard.com

Via E-mail:

webmaster@etextwizard.com
Order Online:

http://www.zipghost.com/order_easychm.html

http://www.etextwizard.com/order_easychm.html

Download URLs:

Supported Operating Systems: Win98,WinME,WinXP,Windows2000,WinVista
System Requirements:
Install Support: Install and Uninstall