Continue to Site

Welcome to

Welcome to our site! is an international Electronics Discussion Forum focused on EDA software, circuits, schematics, books, theory, papers, asic, pld, 8051, DSP, Network, RF, Analog Design, PCB, Service Manuals... and a whole lot more! To participate you need to register. Registration is free. Click here to register now.

what it means by RESOLUTION AND ACCURACY...

Not open for further replies.


Member level 5
Nov 1, 2007
Reaction score
Trophy points
Activity points
resolution and accuracy of meters


i have seen in many digital multimeters and lcr meters that it mentions about resolution and accuracy with ranges. can anybody please help me to get thet answer that what it means by RESOLUTION AND ACCURACY...

let say an example, RANGE = 200pF.....RESOLUTION = 0.1pF....AND ACCURACY = 200pF ~ 200 F (1% rdg + 2 digits)

can anyone please help me to get the answer.

6 1/2 digit resolution calculation

hi dear,

resolution means the least detected value for example if your capacitor is 200 micro farad and the resolution of your maltimeter is .1 micro farad then it will show you values like 200.1 microfarad 200.2 microfarad

accurcy means accurate its denots that the value read by your meter is how much true for examle if any 1 k ohm resistor you are going to read and your meter shows exactly 1 k ohm than tne value by read your meter is 100 percent accurate

resolution andd accuracy of meters

....and the last part about "+2" digits indicates the unit cannot tell you the precise value with the number of digits it can display.

For example: if the actual value is 100.5 and you only have three digits, should the 0.5 be rounded up or down?. Your display of 100 could really be 100.9999 which would indicate almost 1% error but the last digits wont fit on the display.


200pf accuracy

I feel Saharas information is not fully tabled. Accuracy of 1%rdg + 2 digits does not sound scientific. I it could be clarified, more specific and convincing explanations could come.

difference-resolution and accuracy of meters

dear all,

please find the datasheet of item which i m looking for. so that you can have an idea..can you please help me out to find out that the best resolution and best accuracy part of this LCR.

will it measure 1pF or not???

resolution and accuacy in meters

Having seen the data sheet it is seen the information needs explanation !
I too could not get it !


I am also looking some one to explain !

betwixt's explaination seem the right thing but when 10 digits are under suspision, how many digits a 3 1/2 digit display would be displaying.


the difference between accuracy and resolution

In my opinion, the accuray specification in the datasheet is a very common one. Although it's not exact related to measurement technical terms, it's almost sel explanatory. Thus I'm suprized that there's so much guessing about a simple calculation.

Referring to the last example 2mF ~ 20mF (2% rdg + 10 digits)

The displayed value for the said range is 02.00 ... 19.99 mF. The below table shows the
respective relative and absolute error terms:

02.00 19.99 Reading
00.04 00.40 2% of reading
00.10 00.10 "10 digits"
01.86 19.49 Reading - max. error
02.14 20.49 reading + max. error

The apparently confusing "digits" value simply means n*least significant digit respectively n*resolution.

resolution digit means

Thanks FvM
That sounds perfect.
Perhaps the issu arose out of the 3 1/2 and 10 digit.
What I understood is '10 digit is to be taken as 00.10, assuming a 3 1/2 digit display.

resolutins and accuracy in meters

i am really not getting this till date.

my fundamental question was what is the difference and how it says resolution and accuracy.

how can i get the lowest value of measurement of any parameter???

resolutioon and acccuracy of meter

Resolution = the smallest unit that can be measured. In other words, the amount of difference in value between adjacent numbers at the smallest (rightmost) end of the display. This may be different from one measurement range to another.

Accuracy = the amount by which the reading may differ from the real value. Better accuracy means the reading is nearer to the correct value. Absolute reference values are kept at various laboratories around the World and test equipment is generally calibrated through a hierarchy of calibrated equipment to the lab standard.


resolution & accuracy

Sahara could you get it!
betwixt's (and others) do explian properly.
Also 8 digit, 10 digit is standard way of quoting those parameters.
But I am still stuck up with those digits. I tried to explore but failed.

difference resolution accuracy

yeah i got it

even, by defination it is correct and i have already been known.

i am extremely sorry to say that still my questions seems unresolved.

i again repeat my question:-

Accuracy:- +/-(1% rdg + 2 digits)

will this measure 1pF?..if yes, how??

as defination given by betwixt, what will be the answer?


The resolution is telling you the DISPLAY is accurate enough to show 199.8pF, 199.9pF, 200.0pf, 200.1pF and so on.

The accuracy say it is better than +/- 1% plus two counts above or below that figure. So for a value of precisely 200pF, the reading could be up to 1% low (198.0pF) or 1% high (202.0pF) with up to 2 counts error so it could read a minimum of 197.8pF and a maximum of 202.2pF.

Note that the count accuracy is in digits on the display, it doesn't directly equate to an error in measurement, it is caused by rounding the fractions used internally.

So yes, other things being equal, it will measure 1pF and return a reading of between
0.8pF and 1.2pF.


Not open for further replies.

Part and Inventory Search

Welcome to