We are wondering the frequency dependence of FR4 dielectric that effect the characeristic impedance. How much variation in dielectric constant value we can expect in FR4. The highest signal is 10 Gbps.
FR-4 is a minimal standard and doesn't specify RF behaviour beyond 1 MHz. As asked, your question has no meaningful answer.
Most manufacturers have high performance FR-4 alike substrates specifying Er and loss tangent up to 5 or even 10 GHz. They do well support 10 Gbps links, at least up to a certain trace length.
FR-4 just defines Fire Retardant level 4 used by Epoxy-Fiberglass PCB's.
You may want to be aware of Teflon and Ceramic material options.
The skin depth at 10 GHz is smaller than the copper surface roughness p-p , which tells you that path length increases and losses increase as it depends on the conductor surface roughness and thus eddy current losses and impedance. The dielectric loss tangent is another factor that affects dB/cm / GHz losses.