The bit error rate or bit error ratio (BER) is the number of bit errors divided by the total number of transferred bits during a studied time interval. BER is a unitless performance measure, often expressed as a percentage.
The studied time interval is not specified in the problem. So I believe that one byte time interval is not necessary the studied time interval. We just have that BER=0.5.
When a tudied time interval tends to infinity, we have that BER=p=0.5, where p is bit error probability.
We will assume that p=0.5 in this problem.
The probability of finding 3 wrong bits in a byte is the probability of appearing 3 error bits and 5 true bits in a byte simultanuasly multiplied by the number of various ways in which 3 bit out of 8 may be in error.
So:
p=0.5 - error bit probability
(1-p) - true bit probability
p*p*p*(1-p)*(1-p)*(1-p)*(1-p)*(1-p) - the probability of appearing 3 error bits and 5 true bits in a byte simultanuasly
n!/(j!(n-j)!) - is the number of various ways in which j bit out of n may be in error.
And we have:
the probability of finding 3 wrong bits in a byte is:
P=p*p*p*(1-p)*(1-p)*(1-p)*(1-p)*(1-p)*n!/(j!(n-j)!)]=[0.5^8]*[8!/3!/5!]=0.00390625*40320/6/120=0.21875
And the formula given by
Onigece is right