First you need to simulate the full input sweep, picking
off the actual input voltage value at each code transition
and subtracting from it the ideal transition value. Store
all of these (0 - 2^N-1) and use either Ocean code or
an external tool to process the individual bit errors to
get DNL (max()) and INL (I forget, something like
sum()/2^N I guess).
There are ways to make it cleaner in simulation, like
using an ideal DAC off the ADC outputs and a vcvs to
subtract the ADC-DAC voltage from input voltage for
a zero-baseline error signal, that then might be
sampled on timebase, etc. - or at least cleaner for
the analysis point-tool, when that data is printed-to-
file.