I have no idea what is producing the curves you show but it certainly isn't what a ratio detector should give.
A ratio detector works at one specific frequency, it is decided by the capacitors C1 and C2 and L1, L2 and L3 using the formula f=1/(2 *pi *Sqrt(L*C)) and that makes the C1/L1 combination resonate at about 10.9MHz and C2/L2+L3 combination resonate at about 14.7MHz, it therefore doesn't make any sense to show the plot at much lower frequencies. The circuits should resonate at the same frequency, be coupled together and the frequency sweep should be from about 1MHz below resonance to 1MHz above it.
Brian.