To be exact, the term "cutoff wavelength " relates to hollow waveguides, and it is the longest wavelength the waveguide can pass. In microstrip, in reverse, the limiting shortest wavelength is that at which other than TEM mode is propagated, either as above explained, or, radiated by non-TEM modes. This kind of radiation loss is often also affected by inhomogeneities of the microstrip line, like bends, tuning stubs, components and the like.
---------- Post added at 20:42 ---------- Previous post was at 20:40 ----------
By the way, terms of permittivity and permeability do not relate to any antenna but to a dielectric or magnetic material used in it and exposed to electromagnetic waves. So your device is not an antenna but a test jig allowing to measure material properties.