Quantization Noise/Error in an arbitrary system - What new can be done?

[Akanimo]

Hi,

Can quantisation error ever be determined per sample of the quantised signal? Viewing quantisation noise as random noise does not seem to truly solve the problem with quantisation noise.

All responses are welcome , please.

 

[tepalia02]

It is not possible to completely eliminate quantization errors.

 

[crutschow]

Can quantisation error ever be determined per sample of the quantised signal?

You can calculate the theoretical value of the average noise, but not the actual value for a particular sample.
Averaging samples can reduce the error but not eliminate it.

 

[KlausST]

Hi,

are you talking about ADC / DAC?
Or generally in digital signal processing? what data format?

Klaus

 

[Akanimo]

It is not possible to completely eliminate quantization errors.

You are right, because even if it were to be possible to determine quantisation error per sample, it would take an infinite number of bits in the fractional part to completely elimimate it.

--- Updated Dec 1, 2023 ---

You can calculate the theoretical value of the average noise, but not the actual value for a particular sample.
Averaging samples can reduce the error but not eliminate it.

Please kindly state the major reason why it cannot be determined per sample.

--- Updated Dec 1, 2023 ---

Hi,

are you talking about ADC / DAC?
Or generally in digital signal processing? what data format?

Klaus

I'd like to start off with ADC and DAC which should be easy to consider. I would prefer the fixed-point representation.

 

[D.A.(Tony)Stewart]

Quantization errors if perfectly linear will always be 1/2 of the quanta level maximum but could be near 0. It is always measured over a number of bits either as Mean Squared Error or SNR..

Linearity , gain and missing codes are weaknesses of the process measured separately.

The number of samples per sine at the maximum frequency will have quanta interpolation errors which may be measured as distortion compared to the input over a range of cycles depending on the difference frequency. This could be included in SNR vs f for quantization errors of signals in the upper signal frequency range. Sigma Delta quantization technique offers smaller errors with better linearity. with many more bits. There are also other methods of sampling using tertiary encoding with redundancy to reduce quantization errors are UHF speeds.

 

[crutschow]

Please kindly state the major reason why it cannot be determined per sample.

Because it's an unknown depending upon the accuracy of the converter at that particular sample.

 

[D.A.(Tony)Stewart]

Error budget for a typical 8 bit ADC
8 bit ADC 1/2 bit = 0.5 dB error.
Linearity error typ.= 0.25 dB max
Gain error typ. = 0.4 dB max
Offset error = 0.5 dB max
Total 1.65 dB
ENOB = ENOB = (SNR - 1.76) / 6.02. = 7.98 typ. 7.8 min
SNR (dB) = 48.716 dB = 6.02 * N + 1.76dB, where N is the number of effective bits (ENOB).

The single sample quantization error much greater than the RMS error and this does not even include noise..

 

[kaz1]

Hi,

Can quantisation error ever be determined per sample of the quantised signal? Viewing quantisation noise as random noise does not seem to truly solve the problem with quantisation noise.

All responses are welcome , please.

Quantisation noise is inherent and you can only reduce it by either having extra bits or oversampling of a bandlimited signal. As to measuring quantisation error per sample... what for?

 

[Akanimo]

Quantisation noise is inherent and you can only reduce it by either having extra bits or oversampling of a bandlimited signal. As to measuring quantisation error per sample... what for?

If you oversample, that automatically results to additional bits. That is in line with the theory of the day. My aim is to reduce quantisation noise without increasing bit resolution or by even decreasing bit resolution, which is what the sigma-delta modulator does.

--- Updated Dec 7, 2023 ---

Quantization errors if perfectly linear will always be 1/2 of the quanta level maximum but could be near 0. It is always measured over a number of bits either as Mean Squared Error or SNR..

This informs the use of the probability density function and random noise to tackle it. But that is a blackbox approach.

--- Updated Dec 7, 2023 ---

Sigma Delta quantization technique offers smaller errors with better linearity.

True! However, sigma-delta modulators have limitations as to how much they can improve the SQNR. I understand that the sigma-delta modulator shapes quantisation noise by modulation, but they are designed by some kind of blackbox approach. I believe I can achieve more if I can follow an approach that is not blackbox based.

So my principal question is, "As the sigma-delta modulator modulates, what does it do to the signal/quantisation noise that results in the noise being shaped?"

--- Updated Dec 7, 2023 ---

There are also other methods of sampling using tertiary encoding with redundancy to reduce quantization errors are UHF speeds.

I am interested in this. Please suggest articles or other documentation I can look at.

--- Updated Dec 7, 2023 ---

Because it's an unknown depending upon the accuracy of the converter at that particular sample.

Sure, it is an unknown at this time. I don't know how many people believe with me that it can be known. Finding a way to get it known is my primary target right now.

 

[KlausST]

Hi,

so basically there is quantisation noise in ADC and DAC systems.
* The quantisation error ideally is +/- 0.5 LSB.
* there is an additional error by DNL
* and nonlinearities

Now quantisation noise is the frequency spectrum caused by quantisation error over a number over of consecutive samples.

This noise also depends on the true input signal.
Example:
A 12 bit ADC surely has quantisation errors. But when it samples a rather low noise DC signal, there is a good chance that the digital value is constant for let´s say a period of 256 samples.
So now if you perform an FFT over these 256 samples you get a signal at the first bin (DC) but no siganl at all other bins. So no noise at all.

If you do the same on slowly rising (triangle signal) low noise signal you get a clean staircase on the digital side ... maybe with many samples being the same value, followed by several samples with the next higher digital value.
So in the FFT you will see the DC, then (in best case) no noise until the fundamental (digital_value_rise) frequency .. and then agaon nothing than the overtones of the fundamental.

If the input is a clean sine, then I expect DC, the sine as fundamental and overtones to the sine.

This all was for ideal ADC.
On a real ADC you have DNL and nonlinearitiies.
If you do the same tests as above .. you get almost identical FFT results but with increased noise floor.

**
So my answer how to decrease quantisation noise is: (while not increasing bit number, non oversampling, no post noise filtering)
* reduce DNL
* improve linearity
* reduce input frequency (often not possible)
* reduce input signal amplitude (often not possible)
* Or increase input signal amplitude if you want to decrease: signal / quantisation_noise ratio

Klaus

 

[crutschow]

Sure, it is an unknown at this time. I don't know how many people believe with me that it can be known. Finding a way to get it known is my primary target right now.

I would say very few besides yourself believe that, so it seems like a waste of time to explore this further, but it's your time to have fun.

 

[dick_freebird]

I imagine that oversampling and external math post facto, is it.

But that's not "new".

 

[oscillationeer]

If you oversample, that automatically results to additional bits. That is in line with the theory of the day. My aim is to reduce quantisation noise without increasing bit resolution or by even decreasing bit resolution, which is what the sigma-delta modulator does.
...

Sure, it is an unknown at this time. I don't know how many people believe with me that it can be known. Finding a way to get it known is my primary target right now.

Tricks like sigma-delta modulation start with a sample (charge, current) and apply multiple steps to create a number.
The sample, of course, is quantized (has a number of electrons captured on a capacitor, for instance); that is already
a statistically variable quantity, since you cannot capture half an electron.

Thermodynamics means that voltages, currents, and power measurements are always varying. No amount of postprocessing
of a sample will exceed thermal noise limitations on voltages any more than it will exceed quantization limitations on charge.

"Theory of the day" is actually a century or so of theoretical development that is not susceptible to simple suspicions; only
a novel noncompliant observation would challenge those theories.

 

[Akanimo]

Hi,

so basically there is quantisation noise in ADC and DAC systems.
* The quantisation error ideally is +/- 0.5 LSB.
* there is an additional error by DNL
* and nonlinearities

Now quantisation noise is the frequency spectrum caused by quantisation error over a number over of consecutive samples.

This noise also depends on the true input signal.
Example:
A 12 bit ADC surely has quantisation errors. But when it samples a rather low noise DC signal, there is a good chance that the digital value is constant for let´s say a period of 256 samples.
So now if you perform an FFT over these 256 samples you get a signal at the first bin (DC) but no siganl at all other bins. So no noise at all.

If you do the same on slowly rising (triangle signal) low noise signal you get a clean staircase on the digital side ... maybe with many samples being the same value, followed by several samples with the next higher digital value.
So in the FFT you will see the DC, then (in best case) no noise until the fundamental (digital_value_rise) frequency .. and then agaon nothing than the overtones of the fundamental.

If the input is a clean sine, then I expect DC, the sine as fundamental and overtones to the sine.

This all was for ideal ADC.
On a real ADC you have DNL and nonlinearitiies.
If you do the same tests as above .. you get almost identical FFT results but with increased noise floor.

**
So my answer how to decrease quantisation noise is: (while not increasing bit number, non oversampling, no post noise filtering)
* reduce DNL
* improve linearity
* reduce input frequency (often not possible)
* reduce input signal amplitude (often not possible)
* Or increase input signal amplitude if you want to decrease: signal / quantisation_noise ratio

Klaus

I know errors due to differential and integral nonlinearities exist. But I'd like to set those to zero for the purpose of what I am doing.

I think I should have mentioned multi-bit digital sigma-delta modulator instead so that we probably would not have to bother about nonlinearities associated with ADC and DAC.

 

[FvM]

I understand that the sigma-delta modulator shapes quantisation noise by modulation, but they are designed by some kind of blackbox approach. I believe I can achieve more if I can follow an approach that is not blackbox based.

Don't agree about blackbox approach. There's a detailed theory behind delta-sigma modulators.

 

[Akanimo]

Tricks like sigma-delta modulation start with a sample (charge, current) and apply multiple steps to create a number.
The sample, of course, is quantized (has a number of electrons captured on a capacitor, for instance); that is already
a statistically variable quantity, since you cannot capture half an electron.

Thermodynamics means that voltages, currents, and power measurements are always varying. No amount of postprocessing
of a sample will exceed thermal noise limitations on voltages any more than it will exceed quantization limitations on charge.

"Theory of the day" is actually a century or so of theoretical development that is not susceptible to simple suspicions; only
a novel noncompliant observation would challenge those theories.

Yes, I agree that the sigma-delta modulator applies multiple steps to create a number. With the digital sigma-delta modulator, the signal is upsampled to a higher frequency and then interpolated. Then it is quantized to a much lower bit resolution and sampling frequency by downsampling. With feedback, the sigma-delta modulator modulates (i.e. uses multiple step to creat a number, just like you pointed out) at the oversampled frequency. When this numbers are further filtered, we get the noise-shaped signal.

What these digital sigma-delta modulators do at the upsampled data points around each downsampled number is what I believe I can peer into to learn from.

 

[Akanimo]

Don't agree about blackbox approach. There's a detailed theory behind delta-sigma modulators.

Yes, there is a detailed theory behind sigma-delta modulators. However, there is a limit to what SQNR can be achieved. One can't just say I want to design for a SQNR of 300 dB (just to mention a number) and proceed to achieve it, no matter how hard the person tries.

The noise is more or less looked at as random noise. Dither too is random.

 

[KlausST]

Hi,

One can't just say I want to design for a SQNR of 300 dB (just to mention a number) and proceed to achieve it, no matter how hard the person tries.

I agree.
But is the "limit" caused by quantisation noise
or by other noise?

We need to differentiate between them.
In my eyes (without proof) you can not go to 300dB because of other reasons.
Noise adds using square.
let´s say you have 90dB SNR and you add:
* 60dB of SQNR you get --> 60.00dB total. (SNR can completely be ignored)
* 70dB of SQNR you get --> 69.96dB total.
* 80dB of SQNR you get --> 79.59dB total.
* 85dB of SQNR you get --> 83.81dB total.
* 90dB of SQNR you get --> 86.99dB total. (equal influence of SNR and SQNR)
* 95dB of SQNR you get --> 88.81dB total.
* 100dB of SQNR you get --> 89.59dB total.
* 110dB of SQNR you get --> 89.96dB total.
* 120dB of SQNR you get --> 90.00dB total.(SQNR can completely be ignored)

Indeed this is why I don´t "like" those 24 bit delta-sigma ADCs. I see the "24bits" more as a marketing value... because they barely give full 16 bit performance.
(for sure this depends on the exact ADC type).
Maybe their SQNR is way better than 16 bit performance, but the analog and other errors dominate in a way that the true SQNR does not matter at all.

Klaus

 

[D.A.(Tony)Stewart]

The limitation of this method of noise reduction depends on the oversampling ratio and the quality of the loop filter.

Bandwidth and Resolution are known tradeoffs for SD Modulation.

Excess noise outside signal BW must be defined by defining the optimal signal spectrum and model the loop filter or correlator to match it. (Optimal Receiver definition).

http://www.sce.carleton.ca/courses/sysc-3503/w17/notes/rec.pdf