Delay will reduce.
A cell's drive strength indicates how capable they are to charge and discharge the load cap.
More drive strength means more current capability - means faster charging/discharging of load capacitor.
So, resultant slew will be different at the output of a cell [same logic functionality] with different drive strength.
- How much will the delay be reduced or improved ?
-- That depends on load cap and drive strength. If your load cap is insignificantly low, after increasing the drive strength of the cell - you may not see noticeable improvement in the delay, and, so if you have significant cap load, the improvement after replacing the low-drive-strength cell, with a high drive strength will be quite noticeable.