The propagation of TEM (transmission-line) modes comes with many assumptions, a couple of which are a) that the lines are longitudinally invariant (i.e., infinitely long -- obviously this is never the case), and b) that the currents are always balanced in the transverse cross-section of the lines -- which often doesn't happen at various interconnects such as via transitions.
Various components break one or both of these assumptions, so yes, their presence will degrade the quality of the TEM modes. We can state this alternatively as: the TEM mode couples into non-TEM modes at these interfaces, which generally don't act the way we want them to!