there are several factors. Important of them are:
1) Input and output data rate : for example if you are designing any encryptor or decryptor you need minimum 100 MHz;
2) Power: Higher the frequency more the power consumption
3)Accuracy of the results required: If higher accuracy is not needed RC oscillator can be used which saves area... and everything we want in compact size..... but RC cant produce higher frequency !
4) Technology: Lower the node more speed (also more power....again trade off !!).... how much fast we want ?
5) Target platform: Is it FPGA or custom ASIC.... naturally ASIC can give higher clok frequency... but FPGA frequency of operation is limited by several other factors