Gallager...question about "randomly chosen code words&q

[azaz104]

Re: Gallager...question about "randomly chosen code wor

1) In his book, Gallager mentions in section 5.5, page 131, last paragraph:

each code in the ensemble has its own probability of decoding error, assuming maximum likelihood decoding for the code. we shall upper bound the expectation over the ensemble of this error probability. Since at least on code in the ensemble is must have an error probability as small as the ensemble average, this will give us an upper bound on the probability of error for the best code . (please elaborate ?)

2)and on page 134 :
on the other hand, over the ensemble of codes, the two code words will be chosen the same with probability 2^-N ( elaborate here as well)

Added after 2 minutes:

and if someone can open a conversayion concerning this issue in Information theory

 

[bulx]

Re: Gallager...question about "randomly chosen code wor

A variety of coding schemes (an "ensemble") is being considered, without exactly specifying what exactly each coding scheme is. Now each scheme will have a prob of error (assume similar channels for all codes for fair comparison). It looks like there is a way to calculate the average error probability, where the average is calculated across the various coding schemes (ensemble average). If we know that average, we can say that at least one of the coding schemes has prob of error that is equal to or less than this average. So the best of the considered coding schemes must have prob of error less or equal to this average. So, if you know the average, the best code will perform at least as good as it ie; its prob of error should be less than this average number. Thus the average is the upper bound for the best code.

Here it seems a set of binary coding schemes are being considered, with the number of info bits in a code word being N. so 2^N will give you the number of all possible coding schemes. If one coding scheme is randomly chosen, the prob of any scheme being chosen is 1/2^N.
-b

 

[azaz104]

Re: Gallager...question about "randomly chosen code wor

for the last paragraph of your response, there is a set of randomly chosen codes of length N => the single code word probability is 2^-N, I'm OK with that, but in the book, in the limit as ε (cross over error probability) tends to zero, the average error probability tends to 2^-N, how is this?
gallager explains that the probability of choosing the same code word is 2^-N and this is what i don't get