Hi,
I think a scan every 20ms should be enough not to miss even a very short keypress.
*****
Or you can improve:
It seems you want to activate a row by putting sending high to it.
Then a pressed key in that row gives a high at the relating column.
As long as no key is pressed every column is low.
When idle = waiting for a key press, you could activate all rows and set all columns to generate an interrupt when high.
So you don't need to poll all the time, but wait until a column interrupt is generated, Then scan the rows independently as usual.
Klaus