Clock latency just means how much time it took for your clock signal to reach from clock generation point to sink. Ideally, the latency to all the sinks from a clock should be same, meaning all registers are receiving clocks at the same time. However, that is not the case, and hence we add buffers in certain clock paths to make sure the clock reaches all flops at the same time, to make sure that the latency for all the clock paths remain the same. So, to answer your question, yes, it does affect timing